Skip to content

Instantly share code, notes, and snippets.

@zhangw

zhangw/codefuse-chatbot-test-local-llmapi.py

Created February 21, 2024 02:53
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save zhangw/1a6b535c111aae22215d499b7123189d to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save zhangw/1a6b535c111aae22215d499b7123189d to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. zhangw created this gist Feb 21, 2024.
    16 changes: 16 additions & 0 deletions codefuse-chatbot-test-local-llmapi.py
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,16 @@
    # 启动测试
    import openai
    openai.api_key = "" # Not support yet
    openai.api_base = "http://127.0.0.1:18888/v1"

    # 选择你启动的模型
    model = "codellama_34b"

    # create a chat completion
    completion = openai.ChatCompletion.create(
    model=model,
    messages=[{"role": "user", "content": "Hello! What is your name? "}],
    max_tokens=100,
    )
    # print the completion
    print(completion.choices[0].message.content)