Все выпуски

Интерпретируемость моделей глубокого обучения стала центром исследований, особенно в таких областях, как здравоохранение и финансы. Модели с «бутылочным горлышком», используемые для выявления концептов, стали перспективным подходом для достижения прозрачности и интерпретируемости за счет использования набора известных пользователю понятий в качестве промежуточного представления перед слоем предсказания. Однако ручное аннотирование понятий не затруднено из-за больших затрат времени и сил. В нашей работе мы исследуем потенциал больших языковых моделей (LLM) для создания высококачественных банков концептов и предлагаем мультимодальную метрику для оценки качества генерируемых концептов. Мы изучили три ключевых вопроса: способность LLM генерировать банки концептов, сопоставимые с существующими базами знаний, такими как ConceptNet, достаточность унимодального семантического сходства на основе текста для оценки ассоциаций концептов с метками, а также эффективность мультимодальной информации для количественной оценки качества генерации концептов по сравнению с унимодальным семантическим сходством концепт-меток. Наши результаты показывают, что мультимодальные модели превосходят унимодальные подходы в оценке сходства между понятиями и метками. Более того, сгенерированные нами концепты для наборов данных CIFAR-10 и CIFAR-100 превосходят те, что были получены из ConceptNet и базовой модели, что демонстрирует способность LLM генерировать высококачественные концепты. Возможность автоматически генерировать и оценивать высококачественные концепты позволит исследователям работать с новыми наборами данных без дополнительных усилий.

Interpretability in recent deep learning models has become an epicenter of research particularly in sensitive domains such as healthcare, and finance. Concept bottleneck models have emerged as a promising approach for achieving transparency and interpretability by leveraging a set of humanunderstandable concepts as an intermediate representation before the prediction layer. However, manual concept annotation is discouraged due to the time and effort involved. Our work explores the potential of large language models (LLMs) for generating high-quality concept banks and proposes a multimodal evaluation metric to assess the quality of generated concepts. We investigate three key research questions: the ability of LLMs to generate concept banks comparable to existing knowledge bases like ConceptNet, the sufficiency of unimodal text-based semantic similarity for evaluating concept-class label associations, and the effectiveness of multimodal information in quantifying concept generation quality compared to unimodal concept-label semantic similarity. Our findings reveal that multimodal models outperform unimodal approaches in capturing concept-class label similarity. Furthermore, our generated concepts for the CIFAR-10 and CIFAR-100 datasets surpass those obtained from ConceptNet and the baseline comparison, demonstrating the standalone capability of LLMs in generating highquality concepts. Being able to automatically generate and evaluate high-quality concepts will enable researchers to quickly adapt and iterate to a newer dataset with little to no effort before they can feed that into concept bottleneck models.

В данной статье исследуется эффективность применения технологии Retrieval-Augmented Generation (RAG) в сочетании с различными большими языковыми моделями (LLM) для поиска документов и получения информации в корпоративных информационных системах. Рассматриваются варианты использования LLM в корпоративных системах, архитектура RAG, характерные проблемы интеграции LLM в RAG-систему. Предлагается архитектура системы, включающая в себя векторный энкодер текстов и LLM. Энкодер используется для создания векторной базы данных, индексирующей библиотеку корпоративных документов. Запрос, передаваемый LLM, дополняется релевантным ему контекстом из библиотеки корпоративных документов, извлекаемым с использованием векторной базы данных и библиотеки FAISS. Большая языковая модель принимает запрос пользователя и формирует ответ на основе переданных в контексте запроса данных. Рассматриваются общая структура и алгоритм функционирования предлагаемого решения, реализующего архитектуру RAG. Обосновывается выбор LLM для исследования и проводится анализ результативности использования популярных LLM (ChatGPT, GigaChat, YandexGPT, Llama, Mistral, Qwen и др.) в качестве компонента для генерации ответов. На основе тестового набора вопросов методом экспертных оценок оцениваются точность, полнота, грамотность и лаконичность ответов, предоставляемых рассматриваемыми моделями. Анализируются характеристики отдельных моделей, полученные в результате исследования. Приводится информация о средней скорости отклика моделей. Отмечается существенное влияние объема доступной памяти графического адаптера на производительность локальных LLM. На основе интегрального показателя качества формируется общий рейтинг LLM. Полученные результаты подтверждают эффективность предложенной архитектуры RAG для поиска документов и получения информации в корпоративных информационных системах. Были определены возможные направления дальнейших исследований в этой области: дополнение контекста, передаваемого LLM, и переход к архитектуре на базе LLM-агентов. В заключении представлены рекомендации по выбору оптимальной конфигурации RAG и LLM для построения решений, обеспечивающих быстрый и точный доступ к информации в рамках корпоративных информационных систем.

This paper investigates the effectiveness of Retrieval-Augmented Generation (RAG) combined with various Large Language Models (LLMs) for document retrieval and information access in corporate information systems. We survey typical use-cases of LLMs in enterprise environments, outline the RAG architecture, and discuss the major challenges that arise when integrating LLMs into a RAG pipeline. A system architecture is proposed that couples a text-vector encoder with an LLM. The encoder builds a vector database that indexes a library of corporate documents. For every user query, relevant contextual fragments are retrieved from this library via the FAISS engine and appended to the prompt given to the LLM. The LLM then generates an answer grounded in the supplied context. The overall structure and workflow of the proposed RAG solution are described in detail. To justify the choice of the generative component, we benchmark a set of widely used LLMs — ChatGPT, GigaChat, YandexGPT, Llama, Mistral, Qwen, and others — when employed as the answer-generation module. Using an expert-annotated test set of queries, we evaluate the accuracy, completeness, linguistic quality, and conciseness of the responses. Model-specific characteristics and average response latencies are analysed; the study highlights the significant influence of available GPU memory on the throughput of local LLM deployments. An overall ranking of the models is derived from an aggregated quality metric. The results confirm that the proposed RAG architecture provides efficient document retrieval and information delivery in corporate environments. Future research directions include richer context augmentation techniques and a transition toward agent-based LLM architectures. The paper concludes with practical recommendations on selecting an optimal RAG–LLM configuration to ensure fast and precise access to enterprise knowledge assets.

A Large Language Model (LLM) is an advanced artificial intelligence algorithm that utilizes deep learning methodologies and extensive datasets to process, understand, and generate humanlike text. These models are capable of performing various tasks, such as summarization, content creation, translation, and predictive text generation, making them highly versatile in applications involving natural language understanding. Generative AI, often associated with LLMs, specifically focuses on creating new content, particularly text, by leveraging the capabilities of these models. Developers can harness LLMs to automate complex processes, such as extracting relevant information from system requirement documents and translating them into a structured database schema. This capability has the potential to streamline the database design phase, saving significant time and effort while ensuring that the resulting schema aligns closely with the given requirements. By integrating LLM technology with Natural Language Processing (NLP) techniques, the efficiency and accuracy of generating database schemas based on textual requirement specifications can be significantly enhanced. The proposed tool will utilize these capabilities to read system requirement specifications, which may be provided as text descriptions or as Entity-Relationship Diagrams (ERDs). It will then analyze the input and automatically generate a relational database schema in the form of SQL commands. This innovation eliminates much of the manual effort involved in database design, reduces human errors, and accelerates development timelines. The aim of this work is to provide a tool can be invaluable for software developers, database architects, and organizations aiming to optimize their workflow and align technical deliverables with business requirements seamlessly.

A Large Language Model (LLM) is an advanced artificial intelligence algorithm that utilizes deep learning methodologies and extensive datasets to process, understand, and generate humanlike text. These models are capable of performing various tasks, such as summarization, content creation, translation, and predictive text generation, making them highly versatile in applications involving natural language understanding. Generative AI, often associated with LLMs, specifically focuses on creating new content, particularly text, by leveraging the capabilities of these models. Developers can harness LLMs to automate complex processes, such as extracting relevant information from system requirement documents and translating them into a structured database schema. This capability has the potential to streamline the database design phase, saving significant time and effort while ensuring that the resulting schema aligns closely with the given requirements. By integrating LLM technology with Natural Language Processing (NLP) techniques, the efficiency and accuracy of generating database schemas based on textual requirement specifications can be significantly enhanced. The proposed tool will utilize these capabilities to read system requirement specifications, which may be provided as text descriptions or as Entity-Relationship Diagrams (ERDs). It will then analyze the input and automatically generate a relational database schema in the form of SQL commands. This innovation eliminates much of the manual effort involved in database design, reduces human errors, and accelerates development timelines. The aim of this work is to provide a tool can be invaluable for software developers, database architects, and organizations aiming to optimize their workflow and align technical deliverables with business requirements seamlessly.

Large Language Models (LLMs) are transforming software engineering by bridging the gap between natural language and programming languages. These models have revolutionized communication within development teams and the Software Development Life Cycle (SDLC) by enabling developers to interact with code using natural language, thereby improving workflow efficiency. This survey examines the impact of LLMs across various stages of the SDLC, including requirement gathering, system design, coding, debugging, testing, and documentation. LLMs have proven to be particularly useful in automating repetitive tasks such as code generation, refactoring, and bug detection, thus reducing manual effort and accelerating the development process. The integration of LLMs into the development process offers several advantages, including the automation of error correction, enhanced collaboration, and the ability to generate high-quality, functional code based on natural language input. Additionally, LLMs assist developers in understanding and implementing complex software requirements and design patterns. This paper also discusses the evolution of LLMs from simple code completion tools to sophisticated models capable of performing high-level software engineering tasks. However, despite their benefits, there are challenges associated with LLM adoption, such as issues related to model accuracy, interpretability, and potential biases. These limitations must be addressed to ensure the reliable deployment of LLMs in production environments. The paper concludes by identifying key areas for future research, including improving the adaptability of LLMs to specific software domains, enhancing their contextual understanding, and refining their capabilities to generate semantically accurate and efficient code. This survey provides valuable insights into the evolving role of LLMs in software engineering, offering a foundation for further exploration and practical implementation.

Large Language Models (LLMs) are transforming software engineering by bridging the gap between natural language and programming languages. These models have revolutionized communication within development teams and the Software Development Life Cycle (SDLC) by enabling developers to interact with code using natural language, thereby improving workflow efficiency. This survey examines the impact of LLMs across various stages of the SDLC, including requirement gathering, system design, coding, debugging, testing, and documentation. LLMs have proven to be particularly useful in automating repetitive tasks such as code generation, refactoring, and bug detection, thus reducing manual effort and accelerating the development process. The integration of LLMs into the development process offers several advantages, including the automation of error correction, enhanced collaboration, and the ability to generate high-quality, functional code based on natural language input. Additionally, LLMs assist developers in understanding and implementing complex software requirements and design patterns. This paper also discusses the evolution of LLMs from simple code completion tools to sophisticated models capable of performing high-level software engineering tasks. However, despite their benefits, there are challenges associated with LLM adoption, such as issues related to model accuracy, interpretability, and potential biases. These limitations must be addressed to ensure the reliable deployment of LLMs in production environments. The paper concludes by identifying key areas for future research, including improving the adaptability of LLMs to specific software domains, enhancing their contextual understanding, and refining their capabilities to generate semantically accurate and efficient code. This survey provides valuable insights into the evolving role of LLMs in software engineering, offering a foundation for further exploration and practical implementation.