How I Built a GPT-Based Assistant for Our Website

As GPT gains popularity, many businesses are looking to integrate GPT capabilities into their products and services. At OneAI, we wanted to implement a GPT-based assistant on our website to help users with product or API-related queries.

The main challenge here is that GPT isn't aware of our product and API and can't be fine-tuned on our data. We need to provide relevant context and information from our website and documentation for each user query, but how do we select the right information?

{{cta}}

Step 1 - Building the Knowledge-Base

We will build our knowledge-base using OneAI’s Collections API. Collections allow you to store massive amounts of text and cluster them by semantic meaning, without having to set up and configure vector databases. Collections can be queried with natural language, and perform semantic search to get the items that are most matching by meaning. Upon receiving a user question, we will search the collection, and provide GPT with the queried items as context to answer.

```python

import os

import oneai

oneai.api_key = os.getenv("ONEAI_API_KEY")

‍

# build the pipeline

collection_insert = oneai.Pipeline(steps=[

    oneai.skills.HtmlToArticle(), # extract text content from a URL

    oneai.skills.SplitByTopic(), # split the content to smaller chunks

    oneai.skills.CollectionInsert( # insert the chunks into the collection

        input_skill=oneai.skills.SplitByTopic(), # use output of SplitByTopic Skill

        collection="knowledgebase-v1", # collection ID

    ),

])

‍

def upload_to_knowledgebase(urls):

    # run the pipeline

    collection_insert.run_batch(urls)

```

The `upload_to_knowledgebase` function receives a list of URLs, scrapes them for content, and uploads the content into the knowledge-base. The function uses a OneAI pipeline with three Skills:

1. `HtmlToArticle` to extract content from a URL (you can remove this if you already have the text content).
2. `SplitByTopic` to split long pages into smaller chunks. This helps us achieve better granularity when querying the collection, saving token use and removing potentially irrelevant information which might confuse GPT.
3. `CollectionInsert` to insert the results into the knowledge-base collection. Note the `collection` parameter, which is where we specify the collection ID to insert the items into. We will use this same ID to query items later.

You don’t have to configure the collection beforehand, it will get created once you run the pipeline.

‍

<< Download the OneAgent WordPress Plugin >>

‍

Step 2 - Building Prompts

Our prompts to GPT should contain three parts:

1. General instructions
2. The actual user's question
3. The attached context

Let's start with the last part.

Retrieving Context

```python

collection_search = oneai.Pipeline(steps=[

    oneai.skills.CollectionSearch(

         collection="knowledgebase-v1", # same collection ID as upload_to_knowledgebase

         max_phrases=6, # max results to return

    ),

])

‍

def query_knowledgebase(query):

    output = collection_search.run(query).matches

    return output.values

```

The `query_knowledgebase` function uses the `CollectionSearch` Skill to find the most relevant results for a given query.

Writing instructions

Now that GPT has the right context, we need to ensure it replies accurately, responds in the right format, and declines unrelated queries. The instructions we provide in the prompt will significantly impact the quality and reliability of the answers. Unfortunately, there's no set of rules that are guaranteed to work for every task and context. So, you'll have to experiment on your own, and see what yields the best responses. The more you tinker with the prompt, the better the results you’ll get. Taming the GPT beast can be tricky, but here are some helpful tips from my experience working on this project:

• Start the prompt by setting the stage for GPT. Clarify its role, responsibilities, and how users will interact with it. For example, "You are the OneAI assistant, your purpose is to answer user questions on our website".
• Include some general context about your product. Even though we later insert context from our knowledge-base, I found that the responses were better and more brand-compliant when I started with a short paragraph about OneAI and our products. This also helps maintain consistency when no relevant information is found in the knowledge-base.
• Instruct GPT on how to format the response. Be explicit about response length, whether to use markdown formatting, whether to include code blocks, etc. Keep in mind that GPT tends to format answers similarly to the attached knowledge-base info, so you could also influence the response format by adjusting the formatting of your knowledge-base items.
• Explicitly tell GPT to respond only when it's confident in its answer and when the query relates to the provided info, and otherwise, simply say "I'm not sure."
• Clearly indicate which part of the prompt contains the instructions, the user's question, and the attached context. I separated these sections with "========" and added headlines for each part.

```python

def build_prompt(query, context):

    context_str = “\n—--\n”.join(context)

    prompt = f""" 

    You are the OneAI assistant, your purpose is to answer user questions on our website.

    ========

    User question: {query}

    ========

    Information:

    {context_str}

    """ # NOT an actual prompt, just for the code sample

    return prompt

```

Step 3 - Calling GPT

Now, we're all set to send our prompts to GPT.

```python

import openai

openai.api_key = os.get_env("OPENAI_API_KEY")

‍

def send_to_gpt(prompt, temperature=0.5):

    return openai.ChatCompletion.create(

        model="gpt-3.5-turbo",

        messages=[{"role": "user", "content": prompt}],

        temperature=temperature,

        stream=True,

    )

```

This function accepts a prompt and sends it to GPT. Additionally, it takes a `temperature` parameter. Basically, `temperature` controls the randomness of the response. Higher values will result in more “creative” output, while lower values will result in more deterministic responses. I recommend playing a bit with the temperature and discovering which values provide the best responses. 

Note that we use OpenAI's streaming API by setting `stream=True`. This causes the `ChatCompletion.create` method to return a `Generator` object instead of a single response. This means that instead of waiting for the full response, we'll receive partial responses, each containing a piece of the full response, as soon as the model generates them - similar to how ChatGPT works.

Read more about `temperature` and `ChatCompletion.create` here.

Step 4 - Bringing It All Together

```python

def ask_gpt(query):

    # Search the knowledge-base for relevant items

    context = query_knowledgebase(query)

    # Construct the prompt

    prompt = build_prompt(query, context)

    # Invoke GPT

    response = send_to_gpt(prompt)

    # Stream the response

    yield from response

```

In this code, we first search the knowledge-base for relevant information, then build a prompt that combines the query, context, and instructions. Afterward, we call GPT with the prompt and stream partial responses back to the user.

Our `ask_gpt` function can be utilized like this:

```python

query = input("Ask GPT anything about OneAI…")

response = ask_gpt(query)

for partial in response:

    delta = partial.choices[0]["delta"]

    if "content" in delta:

        print(delta["content"], end="")

```

This snippet calls `ask_gpt` and prints partial responses as they arrive.

Step 5 - Improvements

Of course, the provided code serves as a basic skeleton for the actual project, for demonstration purposes. There are many improvements we can relatively easily introduce to our project, such as:

Enable multiple-message Chat

Currently, our code only supports a single question-and-answer interaction, but with trivial changes we can enable longer conversations. All we have to do is to modify the `send_to_gpt` function to accept a list of messages instead of just the initial prompt. Every time the user sends a message, we can attach the entire conversation to the GPT request, providing it with the context of what has been discussed so far. This approach is similar to the way ChatGPT works - it doesn’t actually remember your conversations, but rather receives the entire conversation history each time.

Read more about format of chat completions.

Knowledge-base format

One of the best ways to control the quality and format of GPT responses, is to format the knowledge-base information we attach to the prompt. In the code above, we simply scraped our website and uploaded the content in chunks, without any modifications. Alternatively, we can further process the content before uploading it to our collection. For example, we can use GPT to generate questions and answers based on the content, and upload these to the collection. Here’s a modified `upload_to_knowledgebase` function that accomplishes just that:

```python

 generate_questions_from_url = oneai.Pipeline(steps=[

    oneai.skills.HtmlToArticle(), # extract text content from a URL

    oneai.skills.GPT(

        engine="gpt-3.5-turbo",

        input_skill=oneai.skills.HtmlToArticle(),

        prompt="Based on the following content from OneAI’s website, write a list of addressed questions and answers taken from the content. Response format: …",

        temperature=0.6,

        gpt_api_key=openai.api_key,

    ),

])

collection_insert_2 = oneai.Pipeline(steps=[

    oneai.skills.CollectionInsert(collection="knowledgebase-v1"),

])

‍

def upload_to_knowledgebase_2(urls):

    outputs = generate_questions_from_url.run_batch(urls)

    for input, output in outputs.items():

        completion = output.gpt_text.text

        questions_and_answers = # parse completion into a list of strings, according to the format you specified

        collection_insert_2.run_batch(questions_and_answers)

```

Unlock more capabilities using Collection metadata

The Collections API allows you to attach metadata to collection items, which is then retrieved when searching the collection. This allows you to associate additional data with items, which will tell you more about the user's query when searching the collection. This can power many capabilities in your project, such as:

• For each webpage inserted into the collection, we can store its URL as metadata. After searching the collection, we can offer the user a list of relevant sources for further reading.
• Based on the type of question the user asks, we might want to provide GPT with different prompts. For instance, if the user poses a highly technical question with code elements, we might want to use a different prompt than for general, product-related questions. The items that are found in the collection search may help us determine the type of question the user asked, and select a prompt accordingly. To achieve this, we can add a "type" metadata to each item in the knowledge-base, and then select the prompt based on the "type" with the most occurrences.

To attach metadata to an input, we need to wrap the input string in a `oneai.Input` object as follows:

```python

input1 = oneai.Input("some text or URL", metadata={"type": "technical-question"})

upload_to_knowledgebase([input1, …])

```

Conclusion

And there you have it! I hope this has been helpful in demonstrating how to build a GPT-based assistant. 

There are many ways to further enhance and tailor your AI assistant to suit your specific needs. If you have any questions, concerns, or run into any issues, don't hesitate to reach out to us.