做药的常用网站,株洲网站建设,html网站架设,做h5小程序的网站说明
做这个的主要目的是为了搭建Langchain的本地环境#xff0c;使用LangChain让LLM具备调用自定义函数的功能。
内容
1 安装server
以下将ollama的安装方式#xff0c;以及使用做一个简单的说明(记录#xff09;。之前对这个工具没有了解#xff0c;只是从快速实践的…说明
做这个的主要目的是为了搭建Langchain的本地环境使用LangChain让LLM具备调用自定义函数的功能。
内容
1 安装server
以下将ollama的安装方式以及使用做一个简单的说明(记录。之前对这个工具没有了解只是从快速实践的角度上给到一个参考。
最初是奔着LLM调用自定义函数/API这个功能去的快速看下来一个比较可行的方案是LangChain。
参考页面 后来大致明白 langchain是一个前端的使用包 langserver则是后端服务。先搭好服务然后才能在前端使用。
教程里的几种方式都需要一个大语言后端支持为了避免后续的麻烦所以我决定搭建一下ollama。 然后就跳到了ollama的页面
看起来ollama支持的系统还是比较全面的
1.1 苹果
实测苹果的下载非常快… 秒级。展开后大约435M。 如要使用之前需要在终端上先进行拉取命令就是上面那个。 然后执行测试
import ollama
response ollama.chat(modelllama2, messages[{role: user,content: 解析出收件人地点、公司、收件人和收件人电话\n帮我寄到上海国金中心中心33F, ABC公司Bikky收就行电话号码13566778899。我的电话是18988998899上海杨浦区。,},
])
print(response[message][content])经过分析可以知道* 收件人地点上海市特别是杨浦区
* 公司名称ABC公司
* 收件人姓名Bikky
* 收件人电话号码13566778899
* 你的电话号码18988998899也在上海市但不同区。在执行效率上我的苹果(m1 pro)配置也不算低了但是执行效率上还是比2080TI慢了2~3倍。
mac m1 pro: 平均8秒
[3.5188119411468506,5.56521201133728,7.0565900802612305,11.417732238769531,7.563968896865845,9.987686157226562,6.56359601020813,6.939379930496216,9.785239219665527,11.655837059020996]
In [10]: np.mean(time_list)
Out[10]: 8.0054053544998182080Ti平均2.9秒
[2.193615674972534,2.8509912490844727,2.972665786743164,2.2655117511749268,3.038464069366455,4.976086378097534,3.4014697074890137,2.209334373474121,2.832230567932129,2.2567901611328125]
np.mean(time_list)
2.8997159719467165以下是可拉取的模型最大的70B library
1.2 linux安装
执行脚本
curl -fsSL https://ollama.com/install.sh | sh启动服务
ollama serve执行包的安装
pip3 install ollama -i https://mirrors.aliyun.com/pypi/simple/
pip3 install langchain_community -i https://mirrors.aliyun.com/pypi/simple/
pip3 install beautifulsoup4 -i https://mirrors.aliyun.com/pypi/simple/
pip3 install faiss-cpu -i https://mirrors.aliyun.com/pypi/simple/
pip3 install langchain_text_splitters -i https://mirrors.aliyun.com/pypi/simple/
pip3 install langchain -i https://mirrors.aliyun.com/pypi/simple/然后就可以使用了。 这种方法一般用在没有绝对控制权的情况例如租用的算力主机。
1.3 docker安装
这种方式最为简便也最为有用但是要求对主机具有绝对控制权。
直接拉取最新版本
docker pull ollama/ollama然后启动考虑到server拉取模型会占据很大空间所以把一个大的数据盘挂到root下。ollama下载的文件会存在root的 .ollama隐藏文件夹下面。
docker run -d \--nameollama01 \-v /etc/localtime:/etc/localtime \-v /etc/timezone:/etc/timezone\-v /etc/hostname:/etc/hostname \-v /data:/root \-e LANGC.UTF-8\-p 11434:11434\-w /workspace \--gpus all \ollama/ollama本地docker安装在第一次启动时可能会碰到一些问题
└─ $ docker run -it --rm --gpusall registry.cn-hangzhou.aliyuncs.com/andy08008/pytorch_jupyter:v8 bash
docker: Error response from daemon: could not select device driver with capabilities: [[gpu]].需要做一些连通显卡和docker的操作 1 添加源
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \sudo apt-key add -
distribution$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update2 安装: nvidia-container-toolkit
sudo apt-get install -y nvidia-container-toolkit3 重启通常也就是在这里租用的机器要么没有systemd要么不把这个权限给你。
systemctl restart docker这个服务比较有趣的一点是ollama server会自动进行显存的管理
空闲时 当使用模型预测时
import ollama
response ollama.chat(modelllama2, messages[{role: user,content: 解析出收件人地点、公司、收件人和收件人电话\n帮我寄到上海国金中心中心33F, ABC公司Bikky收就行电话号码13566778899。我的电话是18988998899上海杨浦区。,},
])
print(response[message][content])Based on the information provided, here is a breakdown of the recipients details:* Recipients location: 上海国金中心 (Shanghai Gold Center) in the 33F floor.
* Company name: ABC公司 (ABC Company).
* Recipients name: Bikky收就行 (Bikky receives).
* Recipients phone number: 13566778899.
* Your phone number: 18988998899, located in 上海杨浦区 (Shanghai Yangpu District).Please note that the above information is based on the text you provided and may not be accurate or up-to-date.默认使用llama2-7b模型调用时显存会被占用。 如果过一会不用显存会自动清除。而且如果我们再使用另一个模型时server会自动切换模型把之前的清掉然后载入新的这样显存就不会爆。
import ollama
response ollama.chat(modelllama2:13b, messages[{role: user,content: 解析出收件人地点、公司、收件人和收件人电话\n帮我寄到上海国金中心中心33F, ABC公司Bikky收就行电话号码13566778899。我的电话是18988998899上海杨浦区。,},
])
print(response[message][content])OK! Heres the analysis of the information you provided:1. Receivers location: Shanghai, China (based on the address 上海国金中心中心33F)
2. Company name: ABC Company (based on the address 上海国金中心中心33F)
3. Receivers name: Bikky (based on the name Bikky收)
4. Receivers contact information: Phone number 13566778899 (based on the phone number in the address)
5. Senders location: Shanghai Yangpu District (based on the phone number 18988998899)
6. Senders name: Not providedI hope this helps! Let me know if you have any other questions.2 langchain实验
先快速跟着教程走一遍
1 载入大模型问一个简单的问题 how can langsmith help with testing?
from langchain_community.llms import Ollama
llm Ollama(modelllama2)
llm.invoke(how can langsmith help with testing?)Langsmith is a tool that can be used to test and validate the correctness of language models. Here are some ways in which Langsmith can help with testing:1. **Text generation**: Langsmith can be used to generate text samples that can be used to test the language models ability to produce coherent and contextually relevant text. By comparing the generated text to a reference sample, Langsmith can evaluate the models performance in terms of fluency, coherence, and relevance.
2. **Text completion**: Langsmith can be used to test the language models ability to complete partial sentences or phrases. By providing a starting point for the model and evaluating its output, Langsmith can assess the models ability to capture the context and produce coherent text.
3. **Sentiment analysis**: Langsmith can be used to test the language models ability to classify text as positive, negative, or neutral. By providing a dataset of labeled text samples and evaluating the models performance, Langsmith can assess its ability to accurately classify sentiment.
4. **Named entity recognition**: Langsmith can be used to test the language models ability to identify named entities in text, such as people, organizations, and locations. By providing a dataset of labeled text samples and evaluating the models performance, Langsmith can assess its ability to accurately identify named entities.
5. **Question answering**: Langsmith can be used to test the language models ability to answer questions based on a given context or input. By providing a dataset of labeled question-answer pairs and evaluating the models performance, Langsmith can assess its ability to accurately answer questions.
6. **Dialogue generation**: Langsmith can be used to test the language models ability to engage in coherent and contextually relevant dialogue. By providing a dataset of labeled dialogue samples and evaluating the models performance, Langsmith can assess its ability to produce natural-sounding dialogue.
7. **Multi-task evaluation**: Langsmith can be used to test the language models ability to perform multiple tasks simultaneously, such as language translation, sentiment analysis, and named entity recognition. By providing a dataset of labeled text samples and evaluating the models performance across multiple tasks, Langsmith can assess its ability to handle multi-tasking.By using Langsmith to test these aspects of language models, developers can gain a better understanding of their strengths and weaknesses, and make informed decisions about how to improve them.2 使用prompt模型进行修改
from langchain_core.prompts import ChatPromptTemplate
prompt ChatPromptTemplate.from_messages([(system, You are world class technical documentation writer.),(user, {input})
])3 组成链并使用链提问。感觉风格有点变但内容好坏不确定。
chain prompt | llm
print(chain.invoke({input: how can langsmith help with testing?}))
As a world-class technical documentation writer, I must say that Langsmith is an excellent tool for testing! Here are some ways in which Langsmith can help with testing:1. Automated Testing: Langsmith provides automated testing capabilities, allowing you to create and run tests without any manual intervention. This saves time and effort, while also ensuring consistency and accuracy in your test results.
2. Customizable Tests: With Langsmith, you can create customizable tests tailored to your specific needs. You can define the test cases, question types, and evaluation criteria to suit your requirements.
3. Integration with Existing Tools: Langsmith integrates seamlessly with popular testing tools such as JIRA, TestRail, and PractiTest. This enables you to create, run, and manage tests within a single platform, streamlining your testing process.
4. Collaborative Testing: Langsmith supports collaborative testing, allowing multiple team members to work together on a test project. This fosters collaboration and coordination among team members, ensuring that everyone is on the same page and working towards the same goal.
5. Real-time Feedback: Langsmith provides real-time feedback on test results, enabling you to identify areas of improvement immediately. This helps you to rectify errors early on and ensure that your tests are accurate and reliable.
6. Test Case Management: Langsmith offers a comprehensive test case management system, which enables you to organize, track, and maintain your test cases. This helps you to keep your tests organized, up-to-date, and easily accessible.
7. Reporting and Analytics: Langsmith provides detailed reporting and analytics on test results, allowing you to identify trends, strengths, and weaknesses in your testing process. This enables you to optimize your testing strategy and improve overall quality.
8. Integration with Agile Methodologies: Langsmith is designed to work seamlessly with agile methodologies such as Scrum and Kanban. This ensures that your testing activities are aligned with the rest of your development process, enabling you to deliver high-quality software quickly and efficiently.
9. Customizable Workflows: Langsmith allows you to create customizable workflows tailored to your specific needs. This enables you to streamline your testing process and ensure that it aligns with your projects unique requirements.
10. User-Friendly Interface: Langsmith boasts a user-friendly interface, making it easy for team members to use and navigate the platform. This reduces the learning curve and ensures that everyone can use the platform effectively.In summary, Langsmith is an excellent tool for testing as it provides automated testing capabilities, customizable tests, integration with existing tools, collaborative testing, real-time feedback, test case management, reporting and analytics, integration with agile methodologies, customizable workflows, and a user-friendly interface. These features work together to create a comprehensive and efficient testing platform that can help you deliver high-quality software quickly and efficiently.4 继续加链。应该是对答案做了后除了但也没太看出差别
from langchain_core.output_parsers import StrOutputParseroutput_parser StrOutputParser()
chain prompt | llm | output_parser
print(chain.invoke({input: how can langsmith help with testing?}))As a world-class technical documentation writer, I can help with testing in several ways:1. Content Creation: Langsmith can assist in creating comprehensive and accurate content for your software or system, including user manuals, technical guides, and release notes. This content can be used to test the systems functionality and usability, ensuring that it meets the requirements and expectations of users.
2. Collaborative Testing: Langsmith can collaborate with your testing team to create test cases and scenarios based on the documentation created. This can help ensure that all aspects of the system are thoroughly tested and that any issues or bugs are identified early in the development process.
3. Automated Testing: By using natural language processing (NLP) and machine learning (ML) algorithms, Langsmith can assist in automating testing processes. For example, Langsmith can be used to generate test cases based on the documentation, or to analyze test results and identify areas for improvement.
4. Test Data Generation: Langsmith can help generate test data that is tailored to your systems requirements. This can include generating sample inputs, outputs, and edge cases that can be used to test the systems functionality.
5. Defect Reporting: Langsmith can assist in identifying and reporting defects or issues found during testing. By analyzing the documentation and test results, Langsmith can generate detailed reports of defects and suggest fixes or improvements.
6. Test Planning: Langsmith can help plan and prioritize testing efforts by analyzing the systems requirements and identifying critical areas that need to be tested. This can help ensure that the most important features and functionality are thoroughly tested, and that testing resources are allocated effectively.
7. Test Execution: Langsmith can assist in executing tests by generating test scripts based on the documentation and test cases created. This can help ensure that all aspects of the system are tested and that any issues or bugs are identified early in the development process.
8. Test Data Management: Langsmith can help manage test data by generating sample inputs, outputs, and edge cases that can be used to test the systems functionality. This can help ensure that test data is accurate, up-to-date, and relevant to the system being tested.
9. Performance Tuning: By analyzing the documentation and testing results, Langsmith can assist in identifying performance issues and suggest optimizations or improvements.
10. Security Testing: Langsmith can help identify security vulnerabilities in the system by analyzing the documentation and test results. This can help ensure that the system is secure and meets the necessary security requirements.In summary, Langsmith can assist in various testing activities, including content creation, collaborative testing, automated testing, test data generation, defect reporting, test planning, test execution, test data management, performance tuning, and security testing. By leveraging natural language processing and machine learning algorithms, Langsmith can help improve the efficiency and effectiveness of your testing efforts.5 通过web方式载入数据
from langchain_community.document_loaders import WebBaseLoader
loader WebBaseLoader(https://docs.smith.langchain.com/user_guide)docs loader.load()6 对载入的数据执行分割和嵌入
from langchain_community.embeddings import OllamaEmbeddingsembeddings OllamaEmbeddings()
from langchain_community.vectorstores import FAISS
from langchain_text_splitters import RecursiveCharacterTextSplittertext_splitter RecursiveCharacterTextSplitter()
documents text_splitter.split_documents(docs)
vector FAISS.from_documents(documents, embeddings)7 继续增加链用create_stuff_documents_chain 加上prompt形成文档查询链
from langchain.chains.combine_documents import create_stuff_documents_chainprompt ChatPromptTemplate.from_template(Answer the following question based only on the provided context:context
{context}
/contextQuestion: {input})document_chain create_stuff_documents_chain(llm, prompt)8 载入Document然后执行查询
from langchain_core.documents import Documentdocument_chain.invoke({input: how can langsmith help with testing?,context: [Document(page_contentlangsmith can let you visualize test results)]
})Based on the provided context, Langsmith can help with testing by providing a way to visualize test results. This means that Langsmith can assist in the process of testing software or applications by allowing users to view and analyze the results of those tests in a visual format, such as charts, graphs, or other visualizations.9 然后使用向量增强
from langchain.chains import create_retrieval_chainretriever vector.as_retriever()
retrieval_chain create_retrieval_chain(retriever, document_chain)
response retrieval_chain.invoke({input: how can langsmith help with testing?})
print(response[answer])Based on the provided context, LangSmith can help with testing in several ways:1. Prototyping: LangSmith allows for quick experimentation between prompts, model types, and retrieval strategy, making it easier to understand how the model performs and debug where it is failing.
2. Initial Test Set: LangSmith enables developers to create datasets of inputs and reference outputs, which can be used to run tests on their LLM applications. This helps in identifying regressions with respect to initial test cases.
3. Custom Evaluations: LangSmith provides the ability to run custom evaluations (both LLM and heuristic based) to score test results, allowing developers to assess the performance of their LLM applications more comprehensively.
4. Comparison View: The comparison view in LangSmith enables users to see the results for different configurations side-by-side, helping diagnose regressions in test scores across multiple revisions of the application.
5. Playground Environment: LangSmith provides a playground environment for rapid iteration and experimentation, allowing users to quickly test out different prompts and models without having to run each one individually.这块应该就是很多知识库的一般流程了。
ollama的搭建的使用到这里就结束了。
