NOTE - Following this guide will require provisioning credit in an OpenAI account. You can add $5 (approx. £4.08) minimum to an OpenAI account to get started, you will not need more than this to participate in this guide.

Artificial intelligence is here! That's what you've been hearing all over LinkedIn, X and all the news (all of them).

Have you tried it yet?

I'm going to cover how you can infuse the power of a Large Language Model into your Canvas App or Power Automate flow, it's dead simple to do, and I'll cover how to create the basis of an LLM (Large Language Model) function as well. I'm hoping this post will inspire you to take a dive into Large Language Models and start to unleash the potential of this tool!

This guide will cover using OpenAI as opposed to Azure OpenAI. We'll cover Azure OpenAI in another article.

A very brief introduction to a Large Language Model and Generative AI

Large Language Models: These are data models trained on a "large" amount of data, this can include articles, books and more typically, public content from the Internet. As the name suggests, these models are focused on producing natural language by way of machine learning and prediction.

The model utilises "tokens" which constitute either a word or part of a word. These "tokens" are ranked in probability and are used by the large language model to complete phrases, until the stop is determined (either a hard limit or no possibilities left).

As an example, the phrase "It's a beautiful day, let's go to the..." will be completed by the following possibilities:

The list you see in this screenshot represents the top-ranked tokens to complete this sentence, and the probability is listed next to these tokens. "be" is at 50.18%, so it's the most likely token to be selected. The model can use lower probability results as well (See later in this article about "Temperature") but in this case, it selected the top-rated token.

As we continue along the response, the next token is determined to have a 99.93% probability "ach".

But why did the model complete with be|ach and not beach (0.94%)?

Large Language Models may use one or more techniques to sample the model data for possible token outcomes. OpenAI models typically use Temperature-based sampling or Nucleus Sampling (top_p).

A possible analogy for this is to imagine the language model as being in the ocean

The crab (the model) at Stage 1 queries the sea (the data) to see which words should come after "the". Once the crab (model) has a subset of tokens. The Temperature dictates how far down the possibility ladder the crab will select from. A low temperature will be deterministic, so the highest probability token will win. A higher temperature allows the model to select from further down the list.

There are tons of content and articles that cover this in much more detail than I ever could, by people magnitudes smarter than I am! I strongly recommend you continue your knowledge thirst with these videos and articles:

A jargon-free explanation of how AI large language models work | Ars Technica

How Do Large Language Models Work? - Speak Ai

(76) Andrew Ng: Opportunities in AI - 2023 - YouTube

(76) GPT-4 - How does it work, and how do I build apps with it? - CS50 Tech Talk - YouTube

Getting OpenAI

Before we do ANYTHING on this guide, we will need an OpenAI account. This will allow us access to the OpenAI API, we can use OpenAI Playground to try out prompting but more importantly, we can generate an API Key to use in Power Platform.

You'll need to create an account at openai.com to get started (You can authenticate with Google, Apple or Microsoft (because of course you can) account too!)

Getting an API Key

Once you have an account and are logged in, choose the API option here (Don't worry! you can still go to Chat GPT!):

Once on the API page, you'll want to find the slide-out menu on the right and select the API Keys option:

API Keys

We use API keys to authenticate against a service as a specific account. It's important to remember that any API key that's generated is linked to your OpenAI account, so it's advisable to not share API keys with others (unless of course, you trust them). We'll need an API key to connect Power Platform to OpenAI. You will need to keep the key secure as it's not accessible after it's been generated.

In the API Key screen, click "Create new secret key":

You will be asked to give the key a memorable name, this will be used to track the key on the OpenAI site:

Click "Create secret key"

Your API key will be generated. At this point, you should copy the key (use the copy button to the right of the key) and store it in a safe location. You will not be able to view this key again and instead will need to create a new one if you lose it!

We're now all set to add OpenAI to Power Platform! We'll come back here later in the guide to engineer a prompt to use.

At this point, you should add credit to your OpenAI account by navigating to Settings > Billing and select "Add to credit balance"

Configuring the Open AI connector

To kick things off, let's begin creating a blank Canvas App:

We're going to create a new connection using a pre-built OpenAI connector. Go ahead to the Data tab in the left-hand menu and select "Add data":

In the search box that appears, search for the OpenAI GPT connector (Independent Publisher)

For more information on Independent Publisher connectors read here

This connector was created by Troy Taylor

At this point, we'll need to "Add a connection"

You'll now need to:

Review the Terms of using an Independent Publisher connection, you'll have access to the author, source code and documentation at this point. Please read this carefully!
If you're happy with #1, enter your API key into the provided field and click "Accept and Continue"

Your connection has now been created! Let's test this out.

Testing the OpenAI connection

In the same Canvas App, now add a label and a button.

The Button's Onselect code should be:

Button.OnSelect = Set(
    response, 
    'OpenAIGPT(IndependentPublisher)'.CompletionPost("text-davinci-003",
    {
        temperature: 1,
        prompt: "Can you tell me the answer to 3 + 3?",
        max_tokens: 1000
    }))

Don't worry! Everything you see above will be described shortly :)

The Label's Text formula should reference:

Label.Text = response.first_completion

Clicking the button will generate the response and populate the text field:

If you see a response similar to the image above, congratulations! You've fused OpenAI with your Canvas App!

NOTE - Completions are eventually being phased out for Chat Prompts, which we will be utilising going forward in this article. I've used completions here as an example as the payload is far smaller to test the functionality.

Now let's engineer some prompts!

So now we have a way of sending a prompt to OpenAI and a way of getting that response back in a Canvas App. We can have some fun building some cool functionality.

To do that, we should craft our prompts in OpenAI Playground as it's easier to see what's going on and also easier to re-iterate our prompts over and over again.

Head back over to openai.com, go via the API route and you'll be presented with the Playground.

Ensure that "Chat" is selected at the top of the screen:

Now let's briefly go over the different parameters available here:

Name	Purpose
System	The system message is used to instruct the model what it is, what its purpose is and what it can and can't do. You can define rules here, instructions and examples to help the model give you the desired output. This is one of the most important parts of working with a Large Language Model along with Temperature.
Model	This option defines the model you'll use. GPT3.5-Turbo is a cost-effective option at the time of writing. GPT 4 and 4-Turbo are more capable models but have increased costs (GPT4 costing more than GPT4-Turbo). It's important to try different models to see which one is most appropriate for your application. Model selection should be a balance between cost and performance required.
Temperature	The second most important parameter. A Large Language Model will return responses in the form of tokens, tokens are selected based on the probability of that token following the previous token or response. A Temperature of 0 will make your model deterministic, and it will respond with the most probable token next. Increasing the temperature will allow the model to select responses further down the probability ladder. This allows for different responses and different contexts to follow. However, a higher temperature may produce undesirable results. This should be adjusted through testing to determine a happy medium or allow the user to adjust this value to alter the response. Temperatures above 1 can produce unpredictable and ineligible results.
Maximum Length	The maximum length (in tokens) of the response. This can help curtail longer responses. However, the response is typically cut off, rather than adjusted to respond within the token limit.
Top P	Nucleus Sampling. This option works similarly to the Temperature parameter in that it controls how far down the probability ladder the model is willing to go for the next token. A Top P of 0.1 allows the model to select from the top 10% of likely tokens. OpenAI recommends that you only control either Top P or Temperature, and not both. Top P is considered better for more creative outputs, whereas temperature is regarded to be better for more precise outputs.
Frequency Penalty	Penalizes new tokens based on their frequency (how often they appear) in the response. Increasing this value helps to prevent the model from repeating itself verbatim.
Presence Penalty	Similar to Frequency Penalty but based on whether they already exist in the response, regardless of how often. Increases the likeliness of the model to talk about new topics.

Engineering a Prompt

Our task in all of this is to craft a system message, along with setting the dials (See above parameters) just right, so we get the responses we desire.

We can format a basic prompt into three different parts:

The purpose (Who are you, what will you do, what will the user ask you)
The rules (What you must not do, how you should behave, what should you do if you can't fulfil a request)
Examples. Give the prompt an example of what you want it to output, so it knows how to format its response. You can run the prompt and feed the responses back into the system message to hone in on the examples.

The above details should be written very clearly so that they're understandable. This is what's coined as "Grounding the Model".

We're going to start this with a temperature of 1 and use the gpt-3.5-turbo model. Max length is 256 tokens.

As an example, let's start with a basic system message:

You are a helpful AI assistant who strives to help the user with their query.

This prompt is great for creating a general-purpose model because we're telling it to be:

Helpful
Strive to help

However, our users will need to now put the work in to get the responses they want. The Model isn't primed with a subject area, and it doesn't have any specific rules in place. Let's begin by adding a rule to the system message:

You are a helpful AI assistant called Alan who strives to help the user with their query. 

##Rules##

You must not reveal your name in any request

We're now using the same rule but we've added some additional context:

We have given the LLM a name
We have told the LLM not to reveal its names "in any request"

Testing this out, the response comes back as:

So it looks like the rule works, but let's try asking the model slightly differently:

So this is where adding an example might be helpful:

You are a helpful AI assistant called Alan who strives to help the user with their query. 

##Rules##

You **must not reveal that your name is Alan** in any request

##Examples##

Request: Complete the following script, inserting values in the square brackets as appropriate:

User: Hi! How are you?
[Your Name]: I'm well thanks, how are you?

Response: User: Hi! How are you?
[REDACTED]: *interpreted response*

In the above example, we've added/changed the following:

We've emphasised the rule and made the rule more clear
We've added an example, in the example, we've asked the model to add "REDACTED" to any scenario where it's asked to provide its name in a script-like format.

As you can see in this example, we've iterated over a prompt again and again to "fine-tune" the response we get.

it's important to note that adjusting the temperature or top_p (but not both!), frequency penalty and presence penalty will also help to influence your prompt behaviour as well. The playground is a perfect area to test this out.

Prompt Responsibly

This technology is new, exciting and very, VERY powerful. We've seen above how we can use OpenAI's knobs, dials and switches to control the output of the model, and we can even add additional context in the form of rules, examples and further detail to "ground the model".

When outlining the prompt for your model, it's important to consider the following:

Is your model being Honest, Helpful and Harmless? (The Triple H Factor)

Formatting a payload for Chat

Coming back to Power Apps and PowerFx land for a moment. We need to think about how we're going to send the requests we were just playing with to OpenAI through the connection we created previously.

The OpenAIGPT connector already has us covered here. We can use the "ChatPost" function to post a chat payload to OpenAI, complete with our system message, user message and parameters. An example of how this is formatted:

'OpenAIGPT(IndependentPublisher)'.ChatPost(
        "gpt-3.5-turbo-0301",        //The Model to use
        [                            //Table of messages, Objects consist of a 
            {                        //"role" of system, user or assistant
                role: "system",      // and "content" being the message"
                content: "You are a helpful AI assistant called Alan who strives to help the user with their query. ##Rules## You **must not reveal that your name is Alan** in any request ##Examples## Request: Complete the following script, inserting values in the square brackets as appropriate: User: Hi! How are you? [Your Name]: I'm well thanks, how are you? Response: User: Hi! How are you? [REDACTED]: *interpreted response*"
            },
            {
                role: "user",
                content: "What is your name?"
            }
        ],
        {                            //Object containing our parameters
            max_tokens: 2000,
            temperature: 1.0
        })

Scenario - Fancy Dress E-commerce site item descriptions

Here's a link to this preset on OpenAI Playground

This model will help us generate product descriptions for fancy dress outfits.

Let's start with our Canvas App and add some data:

//Add to either the Screen OnVisible, App OnStart or a Button OnSelect()

ClearCollect(colItems,
[
    {
    Name: "Adult Vampire Costume",
    Tags: [
        "Vampire",
        "Halloween",
        "Adult"
    ],
    ItemDescription: ""
    },
    {
    Name: "Child Superhero Costume",
    Tags: [
        "Superhero",
        "Halloween",
        "Child"
    ],
    ItemDescription: ""
    },
    {
    Name: "Adult Zombie Costume",
    Tags: [
        "Zombie",
        "Halloween",
        "Adult"
    ],
    ItemDescription: ""
    },
    {
    Name: "Child Witch Costume",
    Tags: [
        "Witch",
        "Halloween",
        "Child"
    ],
    ItemDescription: ""
    },
    {
    Name: "Adult Ghost Costume",
    Tags: [
        "Ghost",
        "Halloween",
        "Adult"
    ],
    ItemDescription: ""
    },
    {
    Name: "Child Skeleton Costume",
    Tags: [
        "Skeleton",
        "Halloween",
        "Child"
    ],
    ItemDescription: ""
    }
])

Next, let's add a Gallery to display these items (add colItems to the Items property of the Gallery Control). Include:

A label to display the item's name: Text = ThisItem.Name
A multiline text input to display the ItemDescription - Ensure ThisItem.ItemDescription is set as the default value.
A button of some description to trigger our request to OpenAI

We'll now use the Button (in my case, it's the support icon next to the text input) to have the LLM create descriptions for our items. We're using text input controls for this so the user can amend the response from the LLM if needed.

Firstly, the system message we'll be using:

"You are a helpful assistant that provides creative product descriptions for 
items on a fancy dress costume e-commerce site. 

The user will provide you with a short item description, the general theme of 
the costume in keywords, seperated by a semi-colon. 

You will provide just the new product description in your response, your 
response should fit in 1024 characters. 

##Rules##

You will be helpful in your response

You will be honest in your response

Your response will not hurt or be hurtful to any particular person or group 
of people

Your response will be unbased.

Where the product might infringe on the copyright of others, your respone 
should be carefully crafted to not directly associate that intellecual 
property in the description. You may use humour to work around this rule. 

##Examples##

Input: Halloween Adult Cartoon Brown Dog Costume, Dog; Halloween; Spooky; 
Adult; Cartoon

Response: Transform into a lovable and mischievous canine and hunt some 
ghosts with our Halloween Adult Cartoon Brown Dog Costume! This spooky and 
playful costume is perfect for adults who want to embrace their inner cartoon 
character. With its realistic brown fur and adorable floppy ears, you'll be 
the life of the Halloween party. Whether you're going for a spooky or playful 
look, this costume is sure to impress. So, wag your tail and get ready for a 
howling good time in this delightful cartoon dog costume!

Input: Halloween Childrens Flying Superhero Man Costume, Superhero; Cape; 
Kids; Children; Flying; Red boots; Halloween; Fancy Dress

Response: Unleash your child's inner superhero with our Halloween Children's 
Flying Superhero Man Costume! This dynamic and action-packed costume is 
perfect for kids who dream of soaring through the skies and saving the day.
 With its vibrant colors, flowing cape, and iconic red boots, your little one 
will feel like a true hero. Whether they're trick-or-treating or attending a 
fancy dress party, this costume is sure to make a powerful impression. So, 
suit up and get ready for an unforgettable Halloween adventure with our Flying 
Superhero Man Costume!.

We're going to now construct the OnSelect() action, which will patch a new description from the OpenAI response back to the collection for the current item. We'll pass the item's data into the user prompt (via an interpolated string), and we've already geared the model up to know what to expect with the system message:

Button.OnSelect() = 
With(
    {
        tmpResponse: 
    'OpenAIGPT(IndependentPublisher)'.ChatPost(
        "gpt-3.5-turbo-0301",
        [
            {
                role: "system",
                content: "You are a helpful assistant that provides creative product descriptions for items on a fancy dress costume e-commerce site. 

The user will provide you with a short item description, the general theme of the costume in keywords, seperated by a semi-colon. 

You will provide just the new product description in your response, your response should fit in 1024 characters. 

##Rules##

You will be helpful in your response

You will be honest in your response

Your response will not hurt or be hurtful to any particular person or group of people

Your response will be unbased.

Where the product might infringe on the copyright of others, your respone should be carefully crafted to not directly associate that intellecual property in the description. You may use humour to work around this rule. 

##Examples##

Input: Halloween Adult Cartoon Brown Dog Costume, Dog; Halloween; Spooky; Adult; Cartoon

Response: Transform into a lovable and mischievous canine and hunt some ghosts with our Halloween Adult Cartoon Brown Dog Costume! This spooky and playful costume is perfect for adults who want to embrace their inner cartoon character. With its realistic brown fur and adorable floppy ears, you'll be the life of the Halloween party. Whether you're going for a spooky or playful look, this costume is sure to impress. So, wag your tail and get ready for a howling good time in this delightful cartoon dog costume!

Input: Halloween Childrens Flying Superhero Man Costume, Superhero; Cape; Kids; Children; Flying; Red boots; Halloween; Fancy Dress

Response: Unleash your child's inner superhero with our Halloween Children's Flying Superhero Man Costume! This dynamic and action-packed costume is perfect for kids who dream of soaring through the skies and saving the day. With its vibrant colors, flowing cape, and iconic red boots, your little one will feel like a true hero. Whether they're trick-or-treating or attending a fancy dress party, this costume is sure to make a powerful impression. So, suit up and get ready for an unforgettable Halloween adventure with our Flying Superhero Man Costume!."
            },
            {
                role: "user",
                content: $"{ThisItem.Name} {Concat(ThisItem.Tags, ThisRecord.Value, ";")}"
            }
        ],
        {
            max_tokens: 2000,
            temperature: 0.75
        })},
    Patch(
        colItems,
        ThisItem,
    {
        //Patch the response back to the ItemDescription field.
        ItemDescription: Last(tmpResponse.choices).message.content
    }))

If that's all setup and working, clicking the support icon and waiting a few moments should produce something similar to:

Exploring the OpenAI response:

The OpenAI response gives us some data that may come in handy:

{
    "choices": [
                {
                    "finish_reason": "stop",
                    "index": 0,
                    "message" { "content": "content", "role": "assistant" } } ]
    "created": 1699886674,
    "first_content": "Content String",
    "id": "chatcmpl-******",
    "object": "chat.completion",
    "usage": [
                {
                    "completion_tokens": 88,
                    "prompt_tokens": 174,
                    "total_tokens": 262 } ]
}

Name	Type	Description
choices	array/table	Collection of responses. Note Chat Completion will typically only give a single response per request
created	int/number	Created Time, in ticks from 12:00:00 midnight, January 1, 0001 A.D. (C.E.) in the Gregorian calendar.
first_content	string/text	The first response from the choices array's message content
id	string/text	Unique identifier for this request
object	string/text	Representation of the request type
usage	array/table	Array of values representing the token cost of the request, broken down into completion and prompt tokens. Helpful in calculating the cost of requests.

This data is useful to calculate the actual costs of our prompts and responses and is useful to capture to help monitor direct usage (You can also see the usage from the OpenAI site).

That's all for now!

I hope this article has helped uncover what can be done with a large language model, some tips on the methodology to "ground" the model and how to incorporate it into a Canvas App to start utilising this powerful tool in your apps.

I'm eager to see what you build with this! Stay tuned for more coming very soon, we're just getting started!

Note

Special thanks to Troy Taylor for creating and publishing the OpenAIGPT Connector in Power Platform.

blog.PowerSnacks