LLMverse: A universe dedicated to projects using large language models.
Structure:
- Every project has its entry point under the
projects/folder. - Every project is supported with its own README file which provides relevant information
- Every project comes with its own
requriments.txt, which can be installed in thevirtual environmenton the local machine to achieve full reproducibility of the results.
Table with the projects:
| Project | Short description | Entry Point Link | Supporting material |
|---|---|---|---|
| How Good Are LLMs at Solving Brainteasers? | How do cutting-edge AI models perform on brainteaser questions that demand not only strong reasoning skills but also a touch of creative thinking? In this small study, I evaluate the lateral thinking abilities of frontier OpenAI models, such as GPT-4o and GPT-4o-mini on the dataset with riddles. While the model demonstrated solid performance overall, achieving 84% accuracy, its performance dropped significantly—to 65%—on puzzles that are unlikely to be publicly available on the internet and, therefore, were probably not part of the model’s training data. | link to Notebook with reproducible code | link to article |