PromptingTools.jl supports Groq

PromptingTools.jl supports Groq

PromptingTools.jl, one of my favorite Julia packages for generative AI workflows, now supports groq! For those who do not know, groq is incredibly fast. Of the cloud providers for LLM generation, groq is by far the fastest.

You'll need PromptingTools.jl version 0.22. The release notes are here.

Here's a little demo of how to use this. To start, you'll need a groq API key, which you can find on the website (this link might work?).

Put your key in the environment variable . If you haven't done this at the system level, you can do it inside Julia like so:

Great. Now we can use PromptingTools:

which yielded (for me) the response

Here is an example of Julia code to calculate the n-th Fibonacci number:

This function uses a simple iterative approach to calculate the n-th Fibonacci number. It takes an integer as input and returns the corresponding Fibonacci number.

Here's an explanation of how the code works:

The function takes an integer as input and returns the n-th Fibonacci number. The first two Fibonacci numbers are 0 and 1, so we handle these cases explicitly. For , we use a loop to calculate the n-th Fibonacci number. We initialize two variables and to 0 and 1, respectively, which correspond to the first two Fibonacci numbers. In each iteration of the loop, we update and by swapping their values and adding the previous value of to . This is equivalent to calculating the next Fibonacci number as the sum of the previous two. After iterations, will contain the n-th Fibonacci number, which we return as the result.

You can test this function with a specific value of , for example:

This isn't quite right. Calling this function with yields 34, not 55. This seems to be due to llama3 shifting the function up by one -- should be 0, but here is 0.

But it's close enough for a prompt!

You can also use string macros to make this a bit more concise:

ai

This is in case you're working from the REPL and don't want to type out the function call.

You can use providers that are not groq as well. All providers available in PromptingTools.jl are available here, but the list is quite long. Providers include OpenAI vLLM Ollama Mistral Databricks Fireworks AI Together AI Anthropic Google Gemini

Lastly, if you want to use other model aliases (like ), you can check them out inside :

Anyways -- thanks to Jan for more incredible work!

-- Cameron