{
  "path": "/groq-pt",
  "site": "at://did:plc:gfrmhdmjvxn2sjedzboeudef/site.standard.publication/3md7ylshxzk2y",
  "$type": "site.standard.document",
  "title": "PromptingTools.jl supports Groq",
  "content": {
    "$type": "site.standard.document#markdown",
    "value": "# PromptingTools.jl supports Groq\n\nPromptingTools.jl, one of my favorite Julia packages for generative AI workflows, now supports groq! \nFor those who do not know, groq is [incredibly fast](https://wow.groq.com/groq-sets-new-large-language-model-performance-record-of-300-tokens-per-second-per-user-on-meta-ai-foundational-llm-llama-2-70b/). Of\nthe cloud providers for LLM generation, groq is by far the fastest.\n\nYou'll need PromptingTools.jl version 0.22. The release notes are [here](https://github.com/svilupp/PromptingTools.jl/releases/tag/v0.22.0).\n\nHere's a little demo of how to use this. To start, you'll need a groq API key, which you can find on the website ([this link](https://console.groq.com/keys) might work?).\n\nPut your key in the environment variable `GROQ_API_KEY`. If you haven't done this at the system level, you can do it inside Julia like so:\n\n```julia\nENV[\"GROQ_API_KEY\"] = \"your_key_here\"\n```\n\nGreat. Now we can use PromptingTools:\n\n```julia\nusing PromptingTools\nusing PromptingTools: GroqOpenAISchema\n\n\n# Create the schema\nschema = GroqOpenAISchema()\nsome_julia_code = aigenerate(\n    schema,\n    \"\"\"\n    Give me some Julia code to calculate the n-th Fibonacci number.\n    \"\"\",\n    model=\"gllama370\"\n)\n\n# Show the result\nprintln(some_julia_code.content)\n```\n\nwhich yielded (for me) the response\n\n---\n\nHere is an example of Julia code to calculate the n-th Fibonacci number:\n```\nfunction fibonacci(n::Int)\n    if n == 1\n        return 0\n    elseif n == 2\n        return 1\n    else\n        a, b = 0, 1\n        for i in 3:n\n            a, b = b, a + b\n        end\n        return b\n    end\nend\n```\n\nThis function uses a simple iterative approach to calculate the n-th Fibonacci number. It takes an integer `n` as input and returns the corresponding Fibonacci number.\n\nHere's an explanation of how the code works:\n\n* The function takes an integer `n` as input and returns the n-th Fibonacci number.\n* The first two Fibonacci numbers are 0 and 1, so we handle these cases explicitly.\n* For `n > 2`, we use a loop to calculate the n-th Fibonacci number. We initialize two variables `a` and `b` to 0 and 1, respectively, which correspond to the first two Fibonacci numbers.\n* In each iteration of the loop, we update `a` and `b` by swapping their values and adding the previous value of `a` to `b`. This is equivalent to calculating the next Fibonacci number as the sum of the previous two.\n* After `n-2` iterations, `b` will contain the n-th Fibonacci number, which we return as the result.\n\nYou can test this function with a specific value of `n`, for example:\n\n```julia\njulia> fibonacci(10)\n55\n```\n\n---\n\nThis isn't _quite_ right. Calling this function with `fibonacci(10)` yields 34, not 55. This seems to be due to llama3\nshifting the function up by one -- `fibonacci(0)` should be 0, but here `fibonacci(1)` is 0.\n\nBut it's close enough for a prompt!\n\nYou can also use string macros to make this a bit more concise:\n\n```julia\n\n# Instead, you can also do string macros. You can do this by preceding\n# the string with `ai` and following it with the model you want to use.\n# In this case, we want to use groq's Llama3 70b (gllama370) model.\nai\"Give me some Julia code to calculate the n-th Fibonacci number.\"gllama370\n```\n\nThis is in case you're working from the REPL and don't want to type out the `aigenerate` function call.\n\nYou can use providers that are not groq as well. All providers available in PromptingTools.jl are available [here](https://siml.earth/PromptingTools.jl/dev/coverage_of_model_providers), \nbut the list is quite long. Providers include\n\n- OpenAI\n- vLLM\n- Ollama\n- Mistral\n- Databricks\n- Fireworks AI\n- Together AI\n- Anthropic\n- Google Gemini\n\nLastly, if you want to use other model aliases (like `gllama370`), you can check them out inside `PromptingTools.MODEL_ALIASES`:\n\n```julia\n\njulia> PromptingTools.MODEL_ALIASES\n\nDict{String, String} with 38 entries:\n  \"local\"         => \"local-server\"\n  \"gpt4v\"         => \"gpt-4-vision-preview\"\n  \"gpt3\"          => \"gpt-3.5-turbo\"\n  \"gpt4\"          => \"gpt-4\"\n  \"firefunction\"  => \"accounts/fireworks/models/firefunction-v1\"\n  \"tllama3\"       => \"meta-llama/Llama-3-8b-chat-hf\"\n  \"gpt4t\"         => \"gpt-4-turbo\"\n  \"mistral-tiny\"  => \"mistral-tiny\"\n  \"mistrall\"      => \"mistral-large-latest\"\n  \"emb3small\"     => \"text-embedding-3-small\"\n  \"starling\"      => \"starling-lm\"\n  \"tllama370\"     => \"meta-llama/Llama-3-70b-chat-hf\"\n  \"oh25\"          => \"openhermes2.5-mistral\"\n  \"mistral-large\" => \"mistral-large-latest\"\n  \"gemini\"        => \"gemini-pro\"\n  \"gl3\"           => \"llama3-8b-8192\"\n  \"gllama370\"     => \"llama3-70b-8192\"\n  \"mistralm\"      => \"mistral-medium-latest\"\n  \"tmixtral22\"    => \"mistralai/Mixtral-8x22B-Instruct-v0.1\"\n  \"ollama3\"       => \"llama3:8b-instruct-q5_K_S\"\n  ⋮               => ⋮\n```\n\nAnyways -- thanks to [Jan](https://siml.earth/) for more incredible work!\n\n-- Cameron"
  },
  "publishedAt": "2024-04-21T07:00:00.000Z",
  "textContent": "PromptingTools.jl supports Groq\n\nPromptingTools.jl, one of my favorite Julia packages for generative AI workflows, now supports groq! \nFor those who do not know, groq is incredibly fast. Of\nthe cloud providers for LLM generation, groq is by far the fastest.\n\nYou'll need PromptingTools.jl version 0.22. The release notes are here.\n\nHere's a little demo of how to use this. To start, you'll need a groq API key, which you can find on the website (this link might work?).\n\nPut your key in the environment variable . If you haven't done this at the system level, you can do it inside Julia like so:\n\nGreat. Now we can use PromptingTools:\n\nwhich yielded (for me) the response\n\n---\n\nHere is an example of Julia code to calculate the n-th Fibonacci number:\n\nThis function uses a simple iterative approach to calculate the n-th Fibonacci number. It takes an integer  as input and returns the corresponding Fibonacci number.\n\nHere's an explanation of how the code works:\n\n The function takes an integer  as input and returns the n-th Fibonacci number.\n The first two Fibonacci numbers are 0 and 1, so we handle these cases explicitly.\n For , we use a loop to calculate the n-th Fibonacci number. We initialize two variables  and  to 0 and 1, respectively, which correspond to the first two Fibonacci numbers.\n In each iteration of the loop, we update  and  by swapping their values and adding the previous value of  to . This is equivalent to calculating the next Fibonacci number as the sum of the previous two.\nAfter  iterations,  will contain the n-th Fibonacci number, which we return as the result.\n\nYou can test this function with a specific value of , for example:\n\n---\n\nThis isn't _quite_ right. Calling this function with  yields 34, not 55. This seems to be due to llama3\nshifting the function up by one --  should be 0, but here  is 0.\n\nBut it's close enough for a prompt!\n\nYou can also use string macros to make this a bit more concise:\n\nai\n\nThis is in case you're working from the REPL and don't want to type out the  function call.\n\nYou can use providers that are not groq as well. All providers available in PromptingTools.jl are available here, \nbut the list is quite long. Providers include\nOpenAI\nvLLM\nOllama\nMistral\nDatabricks\nFireworks AI\nTogether AI\nAnthropic\nGoogle Gemini\n\nLastly, if you want to use other model aliases (like ), you can check them out inside :\n\nAnyways -- thanks to Jan for more incredible work!\n\n-- Cameron"
}