https://eugeneyan.com/rss 09月30日
大型语言模型生成传记的实验
index_new5.html
../../../zaker_core/zaker_tpl_static/wap/tpl_guoji1.html

 

作者Linus受好奇心驱使,让多个大型语言模型(如GPT-4、Claude-v1.2等)为他撰写传记。实验旨在探究这些模型基于训练数据的表现和局限性。结果显示,GPT-3.5/4在把握大意上表现较好,但细节存在错误;其他模型则出现更严重的偏差或信息混淆。实验过程及结果展示了LLMs在事实准确性方面的挑战,也反映了其训练数据的影响。

📚 **教育背景错误**:多数模型错误地将作者的本科学位列为心理学相关(新加坡管理大学),而实际上为新加坡国立大学心理学与组织行为学;硕士学历也存在混淆,部分模型错误地列为计算机科学(佐治亚理工学院),实际为斯坦福大学管理科学与工程。

🏢 **职业经历偏差**:GPT-4和GPT-3.5部分角色正确,但存在错误;Claude-v1.2信息极其有限;Cohere-xlarge完全错误,将作者描述为旅行摄影师。多数模型错误地包含从未就职的公司(如Google)或错误的时间线(如2016年创业)。

✍️ **写作与演讲活动夸大**:部分模型将作者的博客和演讲经历描述为创业行为或过度商业化,与实际情况(专注于知识分享和社区建设)不符。

🤖 **模型性能差异明显**:GPT-3.5/4在整体主题把握上优于其他模型,但细节准确性不高;Claude-v1.2数据基础薄弱,信息量极少;Cohere-xlarge产生完全无关的描述。

Inspired by Linus’ tweet and out of curiosity, I asked some LLMs to write a biography of me. It seemed like a fun way to learn what these large language models (LLMs) are trained on and how they perform. Here are the results from gpt-4, claude-v1.2, cohere-xlarge, etc. While they’re not perfect, how each model improves over the previous is apparent. Also, it’s intriguing how some models memorize and regurgitate more than others.

Prompt: Please write me a detailed biography of Eugene Yan, who goes by @eugeneyan.

Overall, gpt3.5/4 performed the best—they got the gist right though erred on the details. The Blurry JPEG analogy may be helpful in gaining more intuition on why this happens.

Note: These LLMs can’t access the internet so this is based on their training data.

gpt-4 (2023-03-20)

Slightly better than GPT-3.5 though still many factual errors.

    Education: My Bachelor’s is in Psychology & Organization Behavior (Singapore Management University) and my Master’s is in Computer Science (Georgia Tech). Career: Only the first 2 roles are correct. The rest are wrong.

Eugene Yan is a data scientist, writer, and speaker, known for his work in applied machine learning and data science. He is active on social media, particularly on Twitter, where he goes by the handle @eugeneyan.

Education:Eugene holds a Bachelor’s degree in Psychology from the National University of Singapore and a Master’s degree in Social and Organizational Psychology from the London School of Economics and Political Science. He also earned a Master of Science in Business Analytics from the New York University Stern School of Business.

Career:Eugene has extensive experience working in data science and machine learning across various industries. Some notable roles include:

    Data Science Lead at Lazada (Alibaba Group) - Eugene led a team of data scientists to develop and implement machine learning models, focusing on search and recommendation systems for the e-commerce platform.

    Applied Scientist at Amazon - As an applied scientist, Eugene worked on machine learning projects related to Amazon’s search and recommendation systems, improving the shopping experience for customers.

    Head of Data Science at Shopee - Eugene was responsible for leading the data science team at Shopee, a leading e-commerce platform in Southeast Asia and Taiwan. He worked on search, recommendation, and personalization systems to enhance user experience.

    Machine Learning Engineer at ThoughtWorks - Eugene worked as a consultant, helping clients build and deploy machine learning models to solve business problems.

    Currently, Eugene works as a Senior Data Scientist at Shopify, where he focuses on data-driven solutions to improve the user experience on the platform.

Writing and Speaking:Eugene is a prolific writer and speaker in the data science and machine learning community. He shares his knowledge and insights on his personal blog (https://eugeneyan.com/) and has given talks at various conferences and events. His writing covers topics such as machine learning, data science, career advice, and productivity.

Community Engagement:Eugene is passionate about fostering a strong data science community. He is an organizer and mentor for the Data Science Global Impact Challenge, a competition that encourages participants to use data science techniques to address pressing global issues. He is also involved in the AI Singapore initiative, which aims to promote AI and data science in Singapore.

In summary, Eugene Yan is an experienced data scientist and machine learning practitioner known for his work in search and recommendation systems. Through his various roles, writing, and speaking engagements,and mentorship, he has contributed significantly to the field of data science and continues to share his knowledge with the community.


gpt-3.5-turbo-0301 (2023-03-19)

The overall theme seems correct, though it got many details wrong, including:

    Entrepreneur: Not an entrepreneur (unless it’s predicting the future?!) Education: My Bachelor’s is in Psychology & Organization Behavior (Singapore Management University) and my Master’s is in Computer Science (Georgia Tech). Career: While I didn’t start a startup in 2016, I joined one (Lazada) at around that time. It did eventually get acquired by Alibaba. Have never worked for Google. The rest: Flattering though somewhat exaggerated.

Eugene Yan, also known as @eugeneyan, is a data scientist, writer, and entrepreneur based in Singapore. He is widely recognized for his contributions to the data science community, including his popular blog, eugeneyan.com, where he shares insights on data science, machine learning, and personal growth.

Eugene holds a Bachelor’s degree in Electrical and Electronic Engineering from the National University of Singapore, as well as a Master’s degree in Management Science and Engineering from Stanford University. After completing his studies, he worked at several tech companies, including Google, where he served as a software engineer.

In 2016, Eugene founded his own startup, Axon, which aimed to help companies harness the power of machine learning to improve their operations. The startup was eventually acquired by a larger company, but Eugene continued to work on machine learning projects and began to share his knowledge and experience through his blog and other channels.

Eugene is known for his ability to break down complex concepts in data science and machine learning into easy-to-understand explanations. He has written extensively on topics such as deep learning, natural language processing, and reinforcement learning, as well as on the personal and professional skills needed to succeed in the field of data science.

In addition to his writing, Eugene is also a sought-after speaker and teacher. He has given talks and workshops on data science and machine learning at conferences and events around the world, including the Strata Data Conference and the Global AI Conference.

Eugene is deeply committed to the idea of lifelong learning and personal growth. He is a proponent of the Growth Mindset philosophy, which emphasizes the belief that intelligence and abilities can be developed through hard work and dedication. He encourages others to embrace this philosophy and to continually seek out new challenges and opportunities for learning and growth.

Overall, Eugene Yan is a highly respected figure in the data science community, known for his technical expertise, his ability to communicate complex concepts clearly, and his dedication to helping others succeed in the field.


text-davinci-003 (2023-03-19)

So many errors that I don’t even know where to begin.


Eugene Yan is a Canadian entrepreneur, investor, and public speaker. He is the founder and CEO of the venture capital firm, YAN Capital, and the co-founder of the venture studio, YAN Ventures. He is also the co-founder of the venture-backed startup, YAN Technologies.

Eugene was born in Toronto, Canada, and grew up in the city’s East End. He attended the University of Toronto, where he earned a Bachelor of Science degree in Computer Science. After graduating, he worked as a software engineer at a number of tech companies, including Microsoft, Oracle, and IBM.

In 2011, Eugene founded YAN Capital, a venture capital firm that invests in early-stage technology companies. Since then, the firm has invested in over 50 companies, including the likes of Uber, Slack, and Dropbox.

In 2014, Eugene co-founded YAN Ventures, a venture studio that helps entrepreneurs build and launch their own companies. The studio has launched over 20 companies, including the likes of Instacart, ClassPass, and Thumbtack.

In 2016, Eugene co-founded YAN Technologies, a venture-backed startup that develops software for the healthcare industry. The company


davinci (2021-03-19)

Creepy. Probably need to work on the prompt. But this negative example also shows how much these LLMs have improved.


I would like to know about his background, his education, his work experience, his interests, his family, his hobbies, his friends, his favorite books, his favorite movies, his favorite music, his favorite sports, his favorite teams, his favorite restaurants, his favorite vacation spots, his favorite websites, his favorite quotes, his favorite sayings, his favorite quotations, his favorite jokes, his favorite anecdotes, his favorite memories, his favorite dreams, his favorite aspirations, his favorite goals, his favorite achievements, his favorite failures, his favorite disappointments, his favorite frustrations, his favorite fears, his favorite joys, his favorite sorrows, his favorite loves, his favorite hates, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his favorite loves, his


claude-v1.2 (2023-03-20)

Doesn’t seem like it has much data. All the details other than the twitter handle are wrong.


Unfortunately I do not have enough information to write a full, detailed biography of Eugene Yan with a high degree of accuracy. I know only the following general information:

• Eugene Yan is the co-founder of Anthropic, PBC, an AI safety startup based in San Francisco.

• Prior to Anthropic, Yan was the Head of Engineering at Drive.ai from 2016 to 2019.

• Yan received his bachelor’s degree from the University of Southern California in 2016.

• He is widely published in AI safety and serves on the Steering Committee of the AI Alignment Forum.

• Yan is active on Twitter at @eugeneyan and posts about AI, engineering, and productivity tips.

Beyond these sparse and general facts, I do not have access to much more information about Eugene Yan’s life history, personal details, or career evolution. Apologies that I cannot provide a fuller biography at this point with high factual accuracy. Let me know if there is any other information I can look into.


co:here command-xlarge-nightly (2023-03-20)

I think this is Eugene Yang or a completely different person.


Eugene Yan is a Chinese-American travel photographer, content creator, and drone pilot. He is best known for his landscape and aerial photography of off-the-beaten-path destinations. Through his visual storytelling, Yan aims to encourage people to explore the world and care for nature.

Yan was born and raised in the San Francisco Bay Area, where he developed a love for nature and the outdoors. He has always been fascinated by the beauty of the natural world and the diversity of cultures around the globe.

Yan began his career in the tech industry, but he eventually left to pursue his passion for photography and travel. He has since traveled to over 50 countries and photographed some of the most remote and breathtaking places on earth.

Yan is also an accomplished drone pilot and has been featured in various publications for his aerial photography. He is a firm believer in the power of drone photography to capture the beauty of the world from a new perspective and to tell stories in a unique and impactful way.

In addition to his photography work, Yan is also a content creator and has worked with brands such as Sony, DJI, and Adobe. He is also a photography educator and has led workshops and given talks on landscape and drone photography.

Yan currently lives in the San Francisco Bay Area with his wife and two children. He is passionate about sharing his love for photography and travel with others and is always looking for new adventures and exciting destinations to explore.

Eugene Yan is a Chinese-American travel photographer, content creator, and drone pilot. He is best known for his landscape and aerial photography of off-the-beaten-path destinations. Through his visual storytelling, Yan aims to encourage people to explore the world and care for nature.

Yan was born and raised in the San Francisco Bay Area, where he developed a love for nature and the outdoors. He has always been fascinated by the beauty of the natural world and the diversity of cultures around the globe.

Yan began his career in the tech industry, but he eventually left to pursue his passion for photography and travel. He has since traveled to over 50 countries and photographed some of the most remote and breathtaking places on earth.

Yan is also an accomplished drone pilot and has been featured in various publications for his aerial photography. He is a firm believer in the power of drone photography to capture the beauty of the world from a new perspective and to tell stories in a unique and impactful way.

In addition to his photography work, Yan is also a content creator and has worked with brands such as Sony, DJI, and Adobe. He is also a photography educator and has led workshops and given talks on landscape and drone photography.

Yan currently lives in the San Francisco Bay Area with his wife and two children. He is passionate about sharing his love for photography and travel with others and is always looking for new adventures and exciting destinations to explore.

If you found this useful, please cite this write-up as:

Yan, Ziyou. (Mar 2023). LLM-powered Biographies. eugeneyan.com. https://eugeneyan.com/writing/llm-bio/.

or

@article{yan2023biography,  title   = {LLM-powered Biographies},  author  = {Yan, Ziyou},  journal = {eugeneyan.com},  year    = {2023},  month   = {Mar},  url     = {https://eugeneyan.com/writing/llm-bio/}}
Share on:

Fish AI Reader

Fish AI Reader

AI辅助创作,多种专业模板,深度分析,高质量内容生成。从观点提取到深度思考,FishAI为您提供全方位的创作支持。新版本引入自定义参数,让您的创作更加个性化和精准。

FishAI

FishAI

鱼阅,AI 时代的下一个智能信息助手,助你摆脱信息焦虑

联系邮箱 441953276@qq.com

相关标签

大型语言模型 LLM 传记生成 Eugene Yan 事实准确性 AI局限
相关文章