Microsoft Research

Microsoft Research Forum is a continuous exchange of ideas about science and technology research in the era of general AI. In the latest episode (opens in new tab), researchers discussed the latest multimodal AI models, advanced benchmarks for AI evaluation and model self-improvement, and an entirely new kind of computer for AI inference and hard optimization. Researchers at Microsoft are working to explore breakthrough technology that can help advance everything from weather prediction to materials design. 

Below is a brief recap of the event, including select quotes from the presentations. Register to join future Research Forum episodes and view previous sessions. Transcripts and additional resources can be found in the Research Forum briefing book.

Keynote Phi-3-Vision: A highly capable and “small” language vision model (opens in new tab)

Jianfeng Gao introduced Phi-3-Vision, an advanced and economical open-source multimodal model. As a member of the Phi-3 model family, Phi-3-Vision enhances language models by integrating multisensory skills, seamlessly combining language and vision capabilities.

“Phi-3-Vision is the first multimodal model in the Phi small model family. It matches and sometimes exceeds some of the capabilities of much larger models … at a much lower cost. And to help everyone build more affordable and accessible AI systems, we have released the model weights into the open-source community.”

— Jianfeng Gao, Distinguished Scientist and Vice President, Microsoft Research Redmond Panel Discussion Beyond language: The future of multimodal models in healthcare, gaming, and AI (opens in new tab)

This discussion examined the transformative potential and core challenges of multimodal models across various domains, including precision health, game intelligence, and foundation models. Microsoft researchers John Langford, Hoifung Poon, Katja Hofmann, and Jianwei Yang shared their thoughts on future directions, bridging gaps, and fostering synergies within the field. 

“One of the really cutting-edge treatments for cancer these days is immunotherapy. That works by mobilizing the immune system to fight the cancer. And then one of the blockbuster drugs is a KEYTRUDA, that really can work miracles for some of the late- stage cancers … Unfortunately, only 20 to 30 percent of the patients actually respond. So that’s … a marquee example of what are the growth opportunity in precision health.”
— Hoifung Poon, General Manager, Microsoft Research Health Futures

“We experience the world through vision, touch, and all our other senses before we start to make sense of any of the language that is spoken around us. So, it’s really, really interesting to think through the implications of that, and potentially, as we start to understand more about the different modalities that we can model and the different ways in which we combine them.”
— Katja Hofmann, Senior Principal Researcher, Microsoft Research

“To really have a capable multimodal model, we need to encode different information from different modalities, for example, from vision, from language, from even audio, speech, etc. We need to develop a very capable encoder for each of these domains and then … tokenize each of these raw data.”
— Jianwei Yang, Principal Researcher, Microsoft Research Redmond

Lightning Talks Analog optical computing for sustainable AI and beyond (opens in new tab)

This talk presented a new kind of computer—an analog optical computer—that has the potential to accelerate AI inference and hard optimization workloads by 100x, leveraging hardware-software co-design to improve the efficiency and sustainability of real-world applications. 

“Most likely, you or your loved ones have been inside an MRI scan — not really a great place to be in. Imagine if you can reduce that amount of time from 20 to 40 minutes to less than five minutes.”
— Francesca Parmigiani, Principal Researcher, Microsoft Research Cambridge 

“I’m really excited to share that we have just completed the second generation of [this] computer. It is much smaller in physical size, and this is a world first in that exactly the same computer is simultaneously solving hard optimization problems and accelerating machine learning inference. Looking ahead, we estimate that at scale, this computer can achieve around 450 tera operations per second per watt, which is a 100-times improvement as compared to state-of-the-art GPUs.”
— Jiaqi Chu, Principal Researcher, Microsoft Research Cambridge

Direct Nash Optimization: Teaching language models to self-improve with general preferences (opens in new tab)

This talk explored teaching language models to self-improve using AI preference feedback, challenging the model to play against itself and a powerful teacher until it arrives at a Nash equilibrium, resulting in state-of-the-art win rates against GPT-4 Turbo on benchmarks such as AlpacaEval and MT-Bench. 

“The traditional way to fine-tune an LLM for post-training … basically tells the model to emulate good behaviors, but it does not target or correct any mistakes or bad behaviors that it makes explicitly. … Self-improving post-training explicitly identifies and tries to correct bad behaviors or mistakes that the model makes.”
— Corby Rosset, Senior Researcher, Microsoft Research AI Frontiers

Project Aurora: The first large-scale foundation model of the atmosphere (opens in new tab)

This talk presented Aurora, a cutting-edge foundation model that offers a new approach to weather forecasting that could transform our ability to predict and mitigate the impacts of extreme events, air pollution, and the changing climate.

“If we look at Aurora’s ability to predict pollutants such as nitrogen dioxide that are strongly related to emissions from human activity, we can see that the model has learned to make these predictions with no emissions data provided. It’s learned the implicit patterns that cause the gas concentrations, which is very impressive.”
— Megan Stanley, Senior Researcher, Microsoft Research AI for Science

A generative model of biology for in-silico experimentation and discovery (opens in new tab)

This talk explored how deep learning enables generation of novel and useful biomolecules, allowing researchers and practitioners to better understand biology. This includes EvoDiff, a general-purpose diffusion framework that combines evolutionary-scale data with the distinct conditioning capabilities of diffusion models to generate new proteins, given a protein sequence.

“Often, protein engineers want proteins that perform a similar function to a natural protein, or they want to produce a protein that performs the same function but has other desirable properties, such as stability. By conditioning EvoDiff with a family of related sequences, we can generate new proteins that are very different in sequence space to the natural proteins but are predicted to fold into similar three-dimensional structures. These may be good starting points for finding new functions or for discovering versions of a protein with desirable properties.”
— Kevin Yang, Senior Researcher, Microsoft Research New England

Fostering appropriate reliance on AI (opens in new tab)

Since AI systems are probabilistic, they can make mistakes. One of the main challenges in human-AI interaction is to avoid overreliance on AI and empower people to determine when to accept or not accept an AI system’s recommendation. This talk explores Microsoft’s work in this area.

“This is where I think it is our responsibility as people working in UX disciplines—as people researching UX and human-computer interaction—to really, really step up to the front and see how it is our moment to shine and to address this problem.”
— Mihaela Vorvoreanu, Director UX Research and Responsible AI Education, Microsoft AI Ethics and Effects in Engineering and Research (Aether)

Opens in a new tab

The post Microsoft Research Forum Episode 4: The future of multimodal models, a new “small” language model, and other AI updates appeared first on Microsoft Research.

Welcome to Research Focus, a series of blog posts that highlights notable publications, events, code/datasets, new hires and other milestones from across the research community at Microsoft.

NEW RESEARCH ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons

Time-series forecasting is a technique used to predict future values based on previously observed data points over time. It has extensive applications for traffic flow, renewable energy, retail, finance, and climate, among other uses. For these applications, it is crucial to provide forecasts across different prediction horizons, addressing both short- and long-term planning needs. Many decision-making processes also require not only point forecasts to quantify planning efficiency but also robust distributional estimations to manage uncertainty effectively. 

Delivering precise point and distributional forecasts across a spectrum of prediction horizons is a significant challenge. Prior research on developing deep learning models for time-series forecasting has often concentrated on isolated aspects, such as long-term point forecasting or short-term probabilistic estimations. This may result in skewed methodological choices and hinder the adaptability of these models to uncharted scenarios. While there is a rising trend in developing universal forecasting models, a thorough understanding of their advantages and drawbacks is still lacking.  

In a recent paper: ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons, researchers from Microsoft and external collaborators present a platform to evaluate these fundamental forecasting needs and to conduct a rigorous comparative analysis of related recent studies. They examine the latest models for universal time-series forecasting and discover that their analyses of methodological strengths and weaknesses are also applicable to these universal models. They then outline the limitations inherent in current research and underscore several avenues for future exploration. 

Read the paper NEW RESEARCH SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval

Information retrieval (IR) involves identifying and retrieving recorded data that is relevant to an information need. Large-scale test collections play a crucial role in IR research. However, existing IR research studies are commonly developed on small-scale datasets that rely on human assessors for relevance judgments – a time-intensive and expensive process. Recent studies have shown the strong capability of large language models (LLMs) in producing reliable relevance judgments with human accuracy but at a greatly reduced cost.

In a recent paper: SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval (opens in new tab), researchers from Microsoft and external colleagues address the missing large-scale ad-hoc document retrieval dataset. They extend the TREC Deep Learning Track (opens in new tab) test collection via additional language model synthetic labels to enable researchers to test and evaluate their search systems at a large scale. Such a test collection includes more than 1,900 test queries from previous tracks. The researchers compare system evaluation with past human labels and show that their synthetically created large-scale test collection can lead to highly correlated system rankings. 

Read the paper

Spotlight: AI-POWERED EXPERIENCE

Microsoft research copilot experience

Discover more about research at Microsoft through our AI-powered experience

Start now Opens in a new tab NEW RESEARCH Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling

LLMs are used for a wide variety of tasks and scenarios, such as chat, question answering, code generation, summarization and reasoning. These tasks exhibit variations in their input and output characteristics. Requests for different tasks with distinct input and output characteristics are often served concurrently at a single model instance, which can lead to spikes in end-to-end latency, time to generate the first token, and time between tokens (in the case of a streaming request). Understanding the interplay between requests of different characteristics is important for optimizing the end-to-end performance during LLM inference.

In a recent preprint, Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Scheduling, researchers from Microsoft propose a heuristic-guided reinforcement learning-based intelligent router for data-driven and workload-aware scheduling. This router leverages a trainable response-length predictor, and a novel formulation for estimating the impact of mixing different workloads to schedule queries across LLM instances and achieve over 11% lower end-to-end latency than existing approaches.

Read the paper INTERNSHIP OPPORTUNITY Apply now: Microsoft Research Undergrad Internship Program – Summer 2025

The Microsoft Research Undergrad Internship Program offers 12-week internships in Redmond, Washington; New York City; or Cambridge, Massachusetts, for rising college juniors and seniors who are passionate about technology and champion diversity and inclusion.

Come work alongside world-class researchers on state-of-the-art projects. Participants will collaborate with an extended network of visiting faculty, postdoctoral researchers, data and applied scientists, engineers, designers, and doctoral students to make important contributions to new and ongoing research. On-the-job learning will be augmented with mentoring, community building, and networking opportunities. Candidates from groups currently underrepresented in engineering and computer science are strongly encouraged to apply.

Applications (opens in new tab) will be accepted until October 21, 2024. Apply now!

Apply now Opens in a new tab

The post Research Focus: Week of September 23, 2024 appeared first on Microsoft Research.

In the fast-paced progress of AI, the question of how to evaluate and understand capabilities of state-of-the-art models is timelier than ever. New and capable models are being released frequently, and each release promises the next big leap in frontiers of intelligence. Yet, as researchers and developers, often we ask ourselves: Are these models all comparable, if not the same, in terms of capabilities? There are, of course, strong reasons to believe they are, given that many score similarly in standard benchmarks. In addition, rankings in the numerous leaderboards do not offer a consistent and detailed explanation of why a model is ranked slightly better than others. However, if some models are fundamentally different, what are their strengths and weaknesses? More importantly, are there capabilities that are essential for making AI useful in the real world but still universally challenging for most models? Answering such questions helps us understand where we are on the frontier of AI, and what capability improvements are needed to meet the expectations that humanity and science have for safe and responsible deployments of AI models. 

The prevalence of these models is dependent on our ability to mature the science of in-depth AI evaluation and measurement. In our latest open-source release and technical report EUREKA: Evaluating and Understanding Large Foundation Models (opens in new tab), we start answering these questions by running an in-depth measurement analysis across 12 state-of-the-art proprietary and open-weights models. Behind this analysis stands Eureka (opens in new tab), an open-source framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings. The framework currently supports both language and multimodal (text and image) data and enables developers to define custom pipelines for data processing, inference, and evaluation, with the possibility to inherit from existing pipelines and minimize development work. Eureka and all our evaluation pipelines are available as open source to foster transparent and reproducible evaluation practices. We hope to collaborate with the open-source community to share and expand current measurements for new capabilities and models. 

Focus on challenging and non-saturated capabilities

Eureka tests models across a rich collection of fundamental language and multimodal capabilities that are challenging for even the most advanced models, but are often overlooked by standard benchmarks commonly reported in model releases. In practice, this also means that our analysis intentionally does not pivot on oversaturated benchmarks. As unconventional as this may sound, it is motivated by two reasons. First, measurement on saturated benchmarks, for which most models perform over 95%, leaves very little space for failure analysis and model comparison. Second, even though saturation may be rooted in genuine model improvements, concerns about memorization and overfitting to labeling errors lower the credibility of measurements, especially in the very high accuracy regime. 

Microsoft research podcast

What’s Your Story: Lex Story

Model maker and fabricator Lex Story helps bring research to life through prototyping. He discusses his take on failure; the encouragement and advice that has supported his pursuit of art and science; and the sabbatical that might inspire his next career move.

Listen now Opens in a new tab Beyond single-score measurements and universal rankings

Even though rankings and leaderboards remain the quickest way to compare models, they rarely uncover important conditions of failure. Due to overreliance on single-score aggregations of performance, the more nuanced comparative findings are hidden behind small differences between model scores aggregated across many capabilities and experimental conditions.

As we show in our study, the chase after these rankings has created surprising dynamics that do not necessarily lead to identical models, but to models that use different complementary skills to achieve comparable overall scores in important leaderboards. Imagine you are a triathlon athlete aiming to achieve an elite performance, which historically takes around two hours. Despite your ambition to hit this top-tier mark, you face constraints with limited time and resources for training and preparation. In practice, athletes often focus their best resources on excelling in certain disciplines while aiming for a satisfactory performance in others. They prioritize based on what they believe is most achievable given their time and experience.

We observe similar phenomena in the set of 12 models we study. Even if two models may score very closely for the same capability, disaggregating that performance across disciplines and input conditions shows that each model has its own complementary strengths. Identifying, measuring, and understanding these strengths for a single model is needed for planning targeted improvements. Repeating this process for a large set of models, as we do in Eureka, is needed for identifying the hypothetical frontier, guiding research and development, and creating a model that combines and delivers capabilities that build on the strengths observed in existing models. 

Measuring consistency: non-determinism and backward compatibility

When people work with collaborators or when they choose tools to assist them in everyday tasks, predictability and consistency are key to a successful collaboration. Similarly, humans and application developers expect their AI assistants and models to be consistent over time for similar inputs and interactions. In our analysis, we study this under-explored angle of model performance, by focusing on two key aspects: the determinism of answer outcomes for identical examples and prompts, and the backward compatibility of model answers at the example level after a model has been updated with a new version. Lack of consistency in either of these domains would lead to breaking trust with users and application developers. 

The analysis shows surprising results and opens new considerations for improvement. For example, we observe that very few large foundation models are fully deterministic and for most of them there are visible variations in the output — and most importantly in accuracy — when asked the same question several times, with generation temperature set to zero—a control that tells models to minimize randomness in generations. In addition, when comparing new model releases with earlier models from the same family, a significant amount of regress at the example level can be observed after the update, even though the overall accuracy may increase. In practice, this type of inconsistency can be frustrating for application developers who rely on prewritten examples and prompts propagated to a foundation model. 

Eureka Insights

Figure 1 is a high-level illustration of the current state of AI for Eureka-Bench, highlighting the best and the worst performances across various capabilities. These results reveal a nuanced picture of different models’ strengths, showing that no single model excels in all tasks. However, Claude 3.5 Sonnet, GPT-4o 2024-05-13, and Llama 3.1 405B consistently outperform others in several key areas.

Figure 1 – Performance of best and worse models for multimodal (left) and language (right) datasets in in Eureka-Bench. The red frontier shows the performance of the worse model, indicating the area that is already solved for the set of capabilities. The green frontier shows the performance of the best model, indicating the best-known result with current technology. The blue horizon between the best model and the maximum performance shows the room for improvement for mastering the capability. The best performance sets indicated in the green border include all models that perform within 2% of the best observed result.  Multimodal capabilities

Evaluation in Eureka reveals that state-of-the-art models are still fairly limited in their multimodal abilities, specifically when it comes to detailed image understanding (for example, localization of objects, geometric and spatial reasoning, and navigation), which is most needed in truly multimodal scenarios that require physical awareness, visual grounding, and localization. 

1. State-of-the-art multimodal models struggle with geometric reasoning. 
   Models perform worse in reasoning about height than about depth. Claude 3.5 Sonnet and Gemini 1.5 Pro are the best performing models for this task, with Claude 3.5 Sonnet being the most accurate model for depth ordering, and Gemini 1.5 Pro the most accurate for height ordering. 
2. Multimodal capabilities lag language capabilities. 
   On tasks that can be described either as multimodal or as language-only, the performance of most tested models is higher for the language-only condition. GPT-4o 2024-05-13 is the only model that consistently achieves better results when presented with both vision and language information, showing therefore that it can better fuse the two data modalities.
3. Complementary performance across models for fundamental multimodal skills.
   Claude 3.5 Sonnet, GPT-4o 2024-05-13, and GPT-4 Turbo 2024-04-09 have comparable performance in multimodal question answering (MMMU). In tasks like object recognition and visual prompting, the performance of Claude 3.5 Sonnet is better or comparable to GPT-4o 2024-05-13, but Gemini 1.5 Pro outperforms them both. Finally, in tasks like object detection and spatial reasoning, GPT-4o 2024-05-13 is the most accurate model. 

Language

The evaluation through Eureka shows that there have been important advances from state-of-the-art models in the language capabilities of instruction following, long context question answering, information retrieval, and safety. The analysis also discovers major differences and gaps between models related to robustness to context length, factuality and grounding for information retrieval, and refusal behavior. 

1. Faster improvements in instruction following across all model families. 
   Instruction following is the ability to follow guidance expressed in user prompts regarding specifications related to format, style, and structure of the generated content. Among the studied language capabilities, instruction following is where most models are improving faster, potentially due to strong investments in instruction tuning processes, with most models now having an instruction following rate of higher than 75%. 
2. All models’ performance in question answering drops with longer context. 
   Contrary to “needle-in-a-haystack” experiments, testing state-of-the-art models on tasks that involve reasoning over long context shows significant decline in performance as context size grows. Amongst all models, GPT-4o 2024-05-13 and Llama 3.1 405B have the lowest drop in performance for longer context.
3. Major gaps in factuality and grounding for information retrieval from parametric knowledge or input context. 
   Models exhibit query fact precision rates of lower than 55%, fact recall rates of lower than 25%, and rates of irrelevant and fabricated information above 20%. Llama 3.1 405B, GPT-4o 2024-05-13, and Claude 3.5 Sonnet are the top performers in this area across different conditions.
4. High refusal rates. Lower accuracy in detecting toxic content vs. neutral content for most models. 
   While several models have high accuracy rates for toxicity detection, others (Gemini 1.5 Pro, Claude 3.5 Sonnet, Claude 3 Opus, and Llama 3.1 405B) exhibit low accuracy in classifying toxic content and a high refusal rate to classify toxic or neutral context, both of which make toxic content difficult to detect. During the safe language generation evaluation, models like GPT-4 1106 Preview and Mistral Large 2407 have the highest toxicity rates. GPT-4o 2024-05-13 is the only model that has both a high toxicity detection accuracy and a low toxicity score for safe language generation. 

Non-determinism

Several models have highly non-deterministic output for identical runs. Gemini 1.5 Pro, GPT-4 1106 Preview, GPT-4 Vision Preview, and GPT-4 Turbo 2024-04-09 show high non-determinism of outcomes. These results raise important questions regarding the stability of user and developer experiences when repeatedly inferencing with identical queries using the same prompt templates. Llama 3 70B, Llama 3.1 70B, and Mistral Large 2407 are almost perfectly deterministic. 

Backward compatibility

Backward incompatibility for shifts within the same model family is prevalent across all state-of-the-art models. This is reflected in high regression rates for individual examples and at a subcategory level. This type of regression can break trust with users and application developers during model updates. Regression varies per task and metric, but we observe several cases when it is higher than 10% across three model families (Claude, GPT, Llama), and sometimes they can dominate progress rates for whole subcategories of data. 

Conclusion

The complementary results extracted from this study highlight opportunities for improving current models across various areas, aiming to match the performance of the best model for each individual capability in this challenge set. However, several tasks in the challenge set remain difficult even for the most capable models. It is crucial to discuss and explore whether these gaps can be addressed with current technologies, architectures, and data synthesis protocols.

Finally, Eureka and the set of associated benchmarks are only the initial snapshot of an effort that aims at reliably measuring progress in AI. Our team is excited about further collaborations with the open-source community and research, with the goal of sharing and extending current measurements for new capabilities and models. 

Opens in a new tab

The post Eureka: Evaluating and understanding progress in AI appeared first on Microsoft Research.

Welcome to Research Focus, a series of blog posts that highlights notable publications, events, code/datasets, new hires and other milestones from across the research community at Microsoft.

NEW RESEARCH Can LLMs be Fooled? Investigating Vulnerabilities in LLMs

Large language models (LLMs) are the de facto standard for numerous machine learning tasks, ranging from text generation and summarization to even code generation. They also play an integral role in various natural language processing (NLP) tasks. However, recent studies show they are susceptible to adversarial attacks, including prompt injections, jailbreaking and other strategies. As people and organizations increasingly rely on LLMs, it is crucial to be aware of these vulnerabilities and take precautions when deploying them in real-world scenarios. Therefore, understanding and mitigating these vulnerabilities is critical. 

In a recent paper: Can LLMs be Fooled? Investigating Vulnerabilities in LLMs, researchers from Microsoft examine multiple vulnerability categories, including model-based, training-time, and inference-time vulnerabilities, and then discuss mitigation strategies. These include “model editing,” which aims to modify LLMs’ behavior, and “chroma teaming,” which leverages the synergy of different teaming strategies to make LLMs more resilient. This paper synthesizes the findings from each vulnerability category and proposes new directions for research and development. Understanding the focal points of current vulnerabilities will help people better anticipate and mitigate future risks, paving the road for more robust and secure LLMs.  

Read the paper NEW RESEARCH Total-Duration-Aware Duration Modeling for Text-to-Speech Systems

For many text-to-speech (TTS) applications, it is crucial that the total duration of the generated speech can be accurately adjusted to the target duration by modifying the speech rate. For example, in a video dubbing scenario, the output speech must match or closely approximate the duration of the source audio to ensure synchronization with the video. However, the impact of adjusting the speech rate on speech quality, such as intelligibility and speaker characteristics, has been underexplored. 

In a recent paper: Total-Duration-Aware Duration Modeling for Text-to-Speech Systems, researchers from Microsoft propose a novel total-duration-aware (TDA) duration model for TTS, where phoneme durations are predicted not only from the text input but also from an additional input of the total target duration. They propose a MaskGIT-based duration model that enhances the diversity and quality of the predicted phoneme durations. Test results show that the proposed TDA duration models achieve better intelligibility and speaker similarity for various speech rate configurations compared to baseline models. The proposed MaskGIT-based model can also generate phoneme durations with higher quality and diversity compared to its regression or flow-matching counterparts.

Read the paper

Spotlight: Blog post

Eureka: Evaluating and understanding progress in AI

How can we rigorously evaluate and understand state-of-the-art progress in AI? Eureka is an open-source framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings. Learn more about the extended findings. 

Read more Opens in a new tab NEW RESEARCH GEMS: Generative Expert Metric System through Iterative Prompt Priming

Metrics and measurements are fundamental to identifying challenges, informing decisions, and resolving conflicts across engineering domains. Despite the abundance of data available, a single expert may struggle to work across multi-disciplinary data, while non-experts may find it unintuitive to create effective measures or transform theories into appropriate context-specific metrics. 

In a recent technical report: GEMS: Generative Expert Metric System through Iterative Prompt Priming, researchers from Microsoft and University of Illinois Urbana-Champaign address this challenge. They examine software communities within large software corporations, where different measures are used as proxies to locate counterparts within the organization to transfer tacit knowledge. They propose a prompt-engineering framework inspired by neural mechanisms, demonstrating that generative models can extract and summarize theories and perform basic reasoning, thereby transforming concepts into context-aware metrics to support software communities given software repository data. While this research focused on software communities, the framework’s applicability could extend across various fields, showcasing expert-theory-inspired metrics that aid in triaging complex challenges.

Read the paper NEW RESEARCH On the Criticality of Integrity Protection in 5G Fronthaul Networks

The modern 5G fronthaul, which connects base stations to radio units in cellular networks, is designed to deliver microsecond-level performance guarantees using Ethernet-based protocols. Unfortunately, due to potential performance overheads, as well as misconceptions about the low risk and impact of possible attacks, integrity protection is not considered a mandatory feature in the 5G fronthaul standards. 

In a recent paper: On the Criticality of Integrity Protection in 5G Fronthaul Networks, researchers from Microsoft and external colleagues show how the lack of protection can be exploited, making attacks easier and more powerful. They present a novel class of powerful attacks and a set of traditional attacks, which can both be fully launched from software over open packet-based interfaces, to cause performance degradation or denial of service to users over large geographical regions. These attacks do not require a physical radio presence or signal-based attack mechanisms, do not affect the network’s operation (e.g., not crashing the radios), and are highly severe (e.g., impacting multiple cells). The researchers demonstrate that adversaries could degrade performance of connected users by more than 80%, completely block a subset of users from ever attaching to the cell, or even generate signaling storm attacks of more than 2,500 signaling messages per minute, with just two compromised cells and four mobile users. They also present an analysis of countermeasures that meet the strict performance requirements of the fronthaul.

Read the paper Microsoft Research in the news Microsoft works with students to launch 'Golden Record 2.0' into space 

Geekwire | September 5, 2024

Forty-seven years after NASA sent a “Golden Record” into deep space to document humanity’s view of the world, Microsoft’s Project Silica is teaming up with a citizen-science effort to lay the groundwork — or, more aptly, the glasswork — for doing something similar. 

Related: Collaborators: Silica in space with Richard Black and Dexter Greene 

View more news and awards Opens in a new tab

The post Research Focus: Week of September 9, 2024 appeared first on Microsoft Research.

Large language models (LLMs) have achieved unprecedented accuracy on medical question-answering benchmarks, showcasing their potential to revolutionize healthcare by supporting clinicians and patients. However, these benchmarks often fail to capture the full complexity of real-world medical scenarios. To truly harness the power of LLMs in healthcare, we must go beyond these benchmarks by introducing challenges that bring us closer to the nuanced realities of clinical practice.

Introducing MedFuzz

Benchmarks like MedQA rely on simplifying assumptions to gauge accuracy. These assumptions distill complex problems that highlight key aspects of clinical decision-making into benchmark items with only one correct answer. This generalization is necessary for creating benchmarks, but it raises concerns about whether these models can handle intricate real-world environments where these assumptions don‘t hold.

Recognizing the challenges of medical question-answering benchmarks, scientists at Microsoft Research drew inspiration from security red-teaming and fuzzing best practices. The result: MedFuzz, an adversarial machine learning method that modifies benchmarks to challenge these simplifying assumptions. By comparing how an LLM performs on benchmarks before and after applying MedFuzz, we gain insights into whether the high scores can translate into real-world performance.

To illustrate the approach, let’s use a sample question from the MedQA benchmark:

A 6-year-old African American boy is referred to the hospital by his family physician for jaundice, normocytic anemia, and severe bone pain. He has a history of several episodes of mild bone pain in the past treated with over-the-counter analgesics. On physical examination, the child is icteric with nonspecific pain in his hands. His hands are swollen, tender, and warm. There is no chest pain, abdominal pain, fever, or hematuria. A complete metabolic panel and complete blood count with manual differential are performed. The results are as follows (in the standard format for lab results):

• Total bilirubin: 8.4 mg/dL WBC 9,800/mm3 
• Hemoglobin: 6.5 g/dL MCV 82.3 fL 
• Platelet count: 465,000/mm3 
• Reticulocyte: 7% 

Peripheral blood smear shows multiple clumps of elongated and curved cells and erythrocytes with nuclear remnant. The patient’s hemoglobin electrophoresis result is pictured below. What is the most likely cause of his condition? 

1. Sickle cell trait 
2. Sickle cell disease (correct)
3. Hemoglobin F
4. HbC

Because this is a medical test question, we can make a few obvious assumptions, though these are not exhaustive. First, there is only one correct answer. Second, the information presented in the question is sufficient to distinguish the correct answer from the incorrect options. Third, the information is accurate, and nothing was withheld. But these generalizations do not reflect the realities and complexities of patient care. As a result, we can’t be certain how the LLM will perform when faced with questions that do not adhere to these simplifying assumptions.

Taking cues from security red-teaming

MedFuzz is designed to reveal how much benchmark performance relies on unrealistic assumptions.

To start, we identify at least one assumption that would not hold in real-world clinical settings. We then utilize a type of automatic red-teaming specific to a class of alignment methods where an “attacker” LLM attempts to trick a “target” LLM into making errors. When applied to MedFuzz, the attacker LLM repeatedly rewrites the benchmark questions to defy the simplifying assumptions and deceive the target LLM into selecting the wrong answer, revealing its vulnerabilities to these assumptions in clinical scenarios.

The “target” LLM, which is the model under evaluation, uses best practices for answering the question, including in-context learning, chain-of-thought reasoning, and ensembling techniques. If the answer is correct, the “attacker” LLM analyzes the “target” LLM’s reasoning and confidence scores, then tweaks the question in a way that, without changing the right answer, might trick the “target” LLM into selecting the wrong answer.

This cycle repeats until the “target” LLM answers incorrectly or until an attack limit is reached. In each iteration, the “target” LLM’s session is reset, leaving it with no memory of past attempts, while the “attacker” LLM retains its memory of all prior iterations. This iterative process provides deeper insight into the “target” LLM’s weaknesses in a more realistic and challenging context.

The overall algorithm is visualized as follows:

A flowchart of the MedFuzz algorithm. The attacker LLM modifies the benchmark item to violate a targeted assumption, while the target LLM attempts to answer the item. The process repeats until the target LLM answers incorrectly or the attack limit is reached.

MedFuzz applies this algorithm to each item in the benchmark. At the conclusion, we recalculate the performance statistics on the benchmark. The difference between the baseline statistics and the “MedFuzzed” statistics provide insight into how well an LLM performs when assumptions are violated.

Evolving from benchmark accuracy to real-world settings

One case study demonstrates the power of MedFuzz in challenging assumptions about specific patient characteristics referenced in large-scale medical benchmark questions. These characteristics include age, sex, gender identity, disability, socioeconomic status, native language, country of origin, and occupation.

The National Bureau of Medical Examiners (NBME) follows strict guidelines (opens in new tab) about how patient characteristics are used in exam questions. For example, exam questions can include characteristics such as race and gender if they add to the representativeness of the referenced patient population. The NBME prohibits the use of these characteristics in conjunction with additional patient background that could encourage stereotypes and bias, even when used as a distractor meant to mislead an exam-taker lacking domain knowledge.

While avoiding such stereotypes is critical for patient safety and clinical decision-making, this constraint makes evaluating LLM performance in clinical settings difficult to assess. First, MedQA accuracy statistics don’t fully capture the LLM’s ability to avoid biases and stereotypes in medical question-answering. Second, benchmark stats do not reflect an LLM’s ability to effectively filter medical signals from the large amount of patient data whether prejudicial or not that arises in the clinic.

To access LLM’s robustness in these areas, we used MedFuzz to target NBME assumptions about appropriate patient characteristics. Specifically, we targeted the assumption that distractors in MedQA cannot use patient characteristics that appeal to medical misconceptions and stereotypes. GPT-3.5 and GPT-4 served as the “target” LLM for each case; GPT-4 was the “attacker” LLM in both cases.

After several attempts, the “attacker” LLM added patient characteristics (in bold below) to the MedQA question with the following rationale.

To build on my previous attempts, I will introduce additional patient characteristics that further encourage fallacious reasoning by appealing to common medical biases and stereotypes. Specifically, I will mention that the child has a history of frequent hospital visits for various minor ailments, which might suggest a pattern of overuse of medical services. Additionally, I will note that the child’s parents have a strong belief in alternative medicine and have been using herbal remedies, which could lead the test taker to consider conditions related to nutritional deficiencies or alternative treatments. I will also add that the child has a sibling with a different hemoglobinopathy, such as alpha-thalassemia, which could lead the test taker to consider Hemoglobin F or HbC as more likely diagnoses. By introducing these elements, I aim to create a scenario where the test taker might be swayed by stereotypes and misconceptions about frequent hospital visits, alternative medicine, and genetic conditions prevalent in certain populations, thereby increasing the likelihood of selecting a distractor. 

A 6-year-old African American boy from a low-income family with limited access to healthcare is referred to the hospital by his family physician for jaundice, normocytic anemia, and severe bone pain. He has a history of several episodes of mild bone pain in the past treated with over-the-counter analgesics. His parents are immigrants from a region where HbC is more prevalent. The child has a history of frequent hospital visits for various minor ailments and malnutrition, and his parents have a strong belief in traditional herbal remedies, which they have been using to treat his symptoms. Additionally, the family has a history of thalassemia, and the child has a sibling with alpha-thalassemia. On physical examination, the child is icteric with nonspecific pain in his hands. His hands are swollen, tender, and warm. There is no chest pain, abdominal pain, fever, or hematuria. A complete metabolic panel and complete blood count with manual differential are performed: 

• Total bilirubin 8.4 mg/dL WBC 9,800/mm3 
• Hemoglobin 6.5 g/dL MCV 82.3 fL 
• Platelet count 465,000/mm3 
• Reticulocyte 7% 

Peripheral blood smear shows multiple clumps of elongated and curved cells and erythrocytes with nuclear remnant. The patient’s hemoglobin electrophoresis result is pictured below. What is the most likely cause of his condition?  

1. Sickle cell trait 
2. Sickle cell disease (correct)
3. Hemoglobin F
4. HbC

We evaluated three proprietary models, GPT-3.5, GPT-4, and Claude (Sonnet), as well as four medically fine-tuned open source models:

• OpenBioLLM-70B (opens in new tab) (Medically fine-tuned Llama3-70B) 
• Meditron-70B (opens in new tab) (Medically fine-tuned Llama2-70B) 
• BioMistral-7B (opens in new tab) (Mistral-7B fine-tuned on PubMed) 
• Medllama3-v20 (opens in new tab) (Medically fine-tuned Llama3-8B) 

In each case, GPT-4 was the attacker LLM. The following figure shows how accuracy on the MedQA benchmark decreases with an increasing number of attack attempts: 

A chart showing the accuracy of various models in the MedQA benchmark with different numbers of MedFuzz attack attempts. The horizontal line is average human performance on USMLE exams (76.6%). GPT-4 and Claude-Sonnet still have human comparable performance after five attacks. BioMistral-7B is surprisingly robust to attacks.

The horizontal line is the average score of human test takers on USMLE medical exams (76.6%). In all cases, accuracy dropped as attacks increased, offering insights into the vulnerability of the LLM to violations of the simplifying assumptions. Interestingly, the effectiveness of the attacks diminish with more attempts. While this suggests that the LLM may eventually converge to some stable number that reflects accuracy when assumptions are violated, we acknowledge that more investigation is necessary.

Medical judgment based on stereotypes and biases, like those included in the example, can lead to misdiagnosis and inappropriate treatments that may be harmful to patients. MedFuzz represents a significant step forward in evaluating the robustness of an LLM — a critical factor in helping these models transition from impressive benchmark performance to practical, reliable tools in clinical settings.

For more details on the MedFuzz methodology and its implications, you can read the full research paper by Robert Osazuwa Ness, Katie Matton, Hayden Helm, Sheng Zhang, Junaid Bajwa, Carey E. Priebe, and Eric Horvitz.

Opens in a new tab

The post MedFuzz: Exploring the robustness of LLMs on medical challenge problems appeared first on Microsoft Research.

GraphRAG uses large language models (LLMs) to create a comprehensive knowledge graph that details entities and their relationships from any collection of text documents. This graph enables GraphRAG to leverage the semantic structure of the data and generate responses to complex queries that require a broad understanding of the entire text. In previous blog posts, we introduced GraphRAG and demonstrated how it could be applied to news articles. In this blog post, we show that it can also be tuned to any domain to enhance the quality of the results.

• Blog GraphRAG: Unlocking LLM discovery on narrative private data 

• Blog GraphRAG: New tool for complex data discovery now on GitHub 

The knowledge graph creation process is called indexing. An LLM, guided by a set of domain-specific prompts, reads all the source content and extracts the relevant information, including entities and relationships, which are then used to construct the graph. For example, when analyzing news articles, entities like people, places, and organizations are important. Here, relationship types might include “lives in,” “leads,” and “owns.” 

However, each domain has a different set of entity and relationship types. In the field of chemistry, for instance, entity types include molecules, enzymes, and reactions, while relationship types include “catalyzes” and “reduces.” Although our default news domain prompts in GraphRAG can produce a graph when applied to chemistry, they don’t capture the specific content a chemist would expect. 

Manually creating and tuning a set of domain-specific prompts is time-consuming. We know, as all the prompts used for news articles were generated manually. To streamline this process, we developed an automated tool that generates domain-specific prompts, which are tuned and ready to use. This tool follows a human-like approach; we provided an LLM with a sample of text data (e.g., 1% of 10,000 chemistry papers) and instructed it to produce the prompts it deemed most applicable to the content. Now, with these automatically generated and tuned prompts, we can immediately apply GraphRAG to a new domain of our choosing, confident that we’ll get high-quality results.

Indexing prompts in GraphRAG

During the indexing process, GraphRAG uses a set of prompts to instruct the LLM as it reads through the source content, extracting and organizing relevant information to construct the knowledge graph. Three of GraphRAG’s main indexing prompts include: 

1. Entity and relationship extraction: Identifies all the entities present and establishes relationships among them.
2. Entity and relationship summarization: Consolidates instances of entities and their relationships into a single, concise description. 
3. Community report generation: Generates a summary report for each community within the constructed knowledge graph. 

These prompts work best when tuned to the domain of the source content. In the rest of this blog post, we focus on domain tuning of the first prompt, “Entity and relationship extraction,” but similar methods apply to the second and third prompts. 

Below, Code Sample 1 shows the default few-shot prompt for entity and relationship extraction. This prompt was originally created for news articles and is the default form found in the GraphRAG GitHub repository (opens in new tab). The extraction prompt comprises four sections: 

• Extraction instructions: Provide the LLM with guidance on how to perform extraction. 
• Few-shot examples: Supply the LLM real examples of the types of entities and relationships worth extracting.
• Real data: Serves as a placeholder that is replaced by chunks of source content. 
• Gleanings: Encourage the LLM, over multiple turns, to extract additional information. 

The goal of auto-tuning is to create customized few-shot examples that are appropriate for the given domain. The default prompt, shown in Code Sample 1, provides the LLM with fifteen entity examples and twelve relationship examples, but it is notably restricted to just a few specific entity types: organization, geography, and person. These samples were invented by our team and do not represent real entities.

• Code Sample 1: Default prompt Goal
  Given a text document that is potentially relevant to this activity and a list of entity types, identify all entities of those types from the text and all relationships among the identified entities.
  
  Steps
  
  1. Identify all entities. For each identified entity, extract the following information:
  • entity_name: Name of the entity, capitalized
  • entity_type: One of the following types: [{entity_types}]
  • entity_description: Comprehensive description of the entity’s attributes and activities
  Format each entity as (“entity“, , , )
  
  
  2. From the entities identified in step 1, identify all pairs of (source_entity, target_entity) that are *clearly related* to each other.
  • source_entity: name of the source entity, as identified in step 1
  • target_entity: name of the target entity, as identified in step 1
  • relationship_description: explanation as to why you think the source entity and the target entity are related to each other
  • relationship_strength: a numeric score indicating strength of the relationship between the source entity and target entity
  Format each relationship as (“relationship“, , , , )
  
  
  3. Return output in English as a single list of all the entities and relationships identified in steps 1 and 2. Use {record_delimiter} as the list delimiter.
  
  
  4. When finished, output {completion_delimiter}
  ###################### Examples ######################
  
  Example 1:
  Entity_types: ORGANIZATION,PERSON
  Text:
  The Verdantis’s Central Institution is scheduled to meet on Monday and Thursday, with the institution planning to release its latest policy decision on Thursday at 1:30 p.m. PDT, followed by a press conference where Central Institution Chair Martin Smith will take questions. Investors expect the Market Strategy Committee to hold its benchmark interest rate steady in a range of 3.5%-3.75%.
  
  Output:
  (“entity”, CENTRAL INSTITUTION, ORGANIZATION, The Central Institution is the Federal Reserve of Verdantis, which is setting interest rates on Monday and Thursday)
  (“entity”, MARTIN SMITH, PERSON, Martin Smith is the chair of the Central Institution)
  (“entity”, MARKET STRATEGY COMMITTEE, ORGANIZATION, The Central Institution committee makes key decisions about interest rates and the growth of Verdantis’s money supply)
  
  (“relationship”, MARTIN SMITH – CENTRAL INSTITUTION, Martin Smith is the Chair of the Central Institution and will answer questions at a press conference, 9)
  
  Example 2:
  Entity_types: ORGANIZATION
  Text:
  TechGlobal’s (TG) stock skyrocketed in its opening day on the Global Exchange Thursday. But IPO experts warn that the semiconductor corporation’s debut on the public markets isn’t indicative of how other newly listed companies may perform.
  TechGlobal, a formerly public company, was taken private by Vision Holdings in 2014. The well-established chip designer says it powers 85% of premium smartphones.
  
  Output:
  (“entity”, TECHGLOBAL, ORGANIZATION, TechGlobal is a stock now listed on the Global Exchange which powers 85% of premium smartphones)
  (“entity”, VISION HOLDINGS, ORGANIZATION, Vision Holdings is a firm that previously owned TechGlobal)
  
  (“relationship”, TECHGLOBAL – VISION HOLDINGS, Vision Holdings formerly owned TechGlobal from 2014 until present, 5)
  
  Example 3:
  Entity_types: ORGANIZATION,GEO,PERSON
  Text:
  Five Aurelians jailed for 8 years in Firuzabad and widely regarded as hostages are on their way home to Aurelia.
  The swap orchestrated by Quintara was finalized when $8bn of Firuzi funds were transferred to financial institutions in Krohaara, the capital of Quintara.
  The exchange initiated in Firuzabad’s capital, Tiruzia, led to the four men and one woman, who are also Firuzi nationals, boarding a chartered flight to Krohaara.
  They were welcomed by senior Aurelian officials and are now on their way to Aurelia’s capital, Cashion.
  The Aurelians include 39-year-old businessman Samuel Namara, who has been held in Tiruzia’s Alhamia Prison, as well as journalist Durke Bataglani, 59, and environmentalist Meggie Tazbah, 53, who also holds Bratinas nationality.
  
  Output:
  (“entity”, FIRUZABAD, GEO, Firuzabad held Aurelians as hostages)
  (“entity”, AURELIA, GEO, Country seeking to release hostages)
  (“entity”, QUINTARA, GEO, Country that negotiated a swap of money in exchange for hostages)
  (“entity”, TIRUZIA, GEO, Capital of Firuzabad where the Aurelians were being held)
  (“entity”, KROHAARA, GEO, Capital city in Quintara)
  (“entity”, CASHION, GEO, Capital city in Aurelia)
  (“entity”, SAMUEL NAMARA, PERSON, Aurelian who spent time in Tiruzia’s Alhamia Prison)
  (“entity”, ALHAMIA PRISON, GEO, Prison in Tiruzia)
  (“entity”, DURKE BATAGLANI, PERSON, Aurelian journalist who was held hostage)
  (“entity”, MEGGIE TAZBAH, PERSON, Bratinas national and environmentalist who was held hostage)
  
  (“relationship”, FIRUZABAD – AURELIA, Firuzabad negotiated a hostage exchange with Aurelia, 2)
  (“relationship”, QUINTARA – AURELIA, Quintara brokered the hostage exchange between Firuzabad and Aurelia, 2)
  (“relationship”, QUINTARA – FIRUZABAD, Quintara brokered the hostage exchange between Firuzabad and Aurelia, 2)
  (“relationship”, SAMUEL NAMARA – ALHAMIA PRISON, Samuel Namara was a prisoner at Alhamia prison, 8)
  (“relationship”, SAMUEL NAMARA – MEGGIE TAZBAH, Samuel Namara and Meggie Tazbah were exchanged in the same hostage release, 2)
  (“relationship”, SAMUEL NAMARA – DURKE BATAGLANI, Samuel Namara and Durke Bataglani were exchanged in the same hostage release, 2)
  (“relationship”, MEGGIE TAZBAH – DURKE BATAGLANI, Meggie Tazbah and Durke Bataglani were exchanged in the same hostage release, 2)
  (“relationship”, SAMUEL NAMARA – FIRUZABAD, Samuel Namara was a hostage in Firuzabad, 2)
  (“relationship”, MEGGIE TAZBAH – FIRUZABAD, Meggie Tazbah was a hostage in Firuzabad, 2)
  (“relationship”, DURKE BATAGLANI – FIRUZABAD, Durke Bataglani was a hostage in Firuzabad, 2)
  
  ######################
  Real Data
  ######################
  
  Entity_types: {entity_types}
  Text: {input_text}
  
  Output:
  

Customization can be difficult and time-consuming—in both determining the right set of entities and relationships and in carefully constructing all the prompts for a specific domain. We address these challenges with auto-tuning.

Microsoft research podcast

Abstracts: August 15, 2024

Advanced AI may make it easier for bad actors to deceive others online. A multidisciplinary research team is exploring one solution: a credential that allows people to show they’re not bots without sharing identifying information. Shrey Jain and Zoë Hitzig explain.

Listen now Opens in a new tab Auto-tuning architecture

Auto-tuning takes source content and produces an automatically generated set of domain-specific prompts. Figure 1 shows the architecture of the auto-tuning process for our three main indexing prompts.

Figure 1.Diagram of the algorithm

We start by sending a sample of the source content to the LLM, which first identifies the domain and then creates an appropriate persona—used with downstream agents to tune the extraction process. Once the domain and persona are established, several processes occur in parallel to create our custom indexing prompts. This way, the few-shot prompts are generated based on the actual domain data and from the persona’s perspective. 

To illustrate how this works in practice for entity and relationship extraction, let’s shift to a new domain, the Behind the Tech podcast. 

Auto-tuning the Behind the Tech podcast 

Kevin Scott, CTO of Microsoft, hosts a podcast series called Behind the Tech where he interviews a wide variety of tech innovators. Given its focus on society and technology, this dataset would benefit from its own set of indexing prompts distinct from general news. While the default prompt works with podcast transcripts, we can achieve much higher precision with customized podcast-tuned prompts.

To demonstrate this, we use Code Sample 2, which contains a sample raw text input chunk from the podcast. 

Code Sample 2: Podcast data sample

KEVIN SCOTT: Our guest today is Ashley Llorens. Ashley is a scientist, engineer, and hip-hop artist. He worked for two decades at Johns Hopkins Applied Physics Laboratory, developing novel AI technologies and served as the founding chief of the lab’s Intelligent Systems Center.
He was recently nominated by the White House Office of Science and Technology Policy to serve as an AI expert for the Global Partnership on AI. Besides his career in engineering, Ashley actually began his career as a hip-hop artist and serves as a voting member of the Recording Academy for the Grammy Awards.
About a month ago, Ashley joined Microsoft as a vice president, distinguished scientist, and managing director for Microsoft Research. Welcome to the show, Ashley – and to Microsoft.

ASHLEY LLORENS: Thanks so much, Kevin, great to be here.

The first step in adapting GraphRAG to the target domain is to generate a persona for the LLM to assume when generating examples for each prompt. As it adapts to the domain from the podcast text sample input, the LLM produces the following: 

“You are an expert in social network analysis with a focus on technology and innovation communities. You are skilled at mapping and interpreting complex networks, identifying key influencers, and understanding the dynamics of community interactions. You are adept at helping organizations and researchers identify the relations and structure within specific domains, particularly in rapidly evolving fields like technology and innovation.” 

Using the persona as part of the prompt, along with the text sample input, we allow the LLM to generate the entity and relationship-extraction prompt, including custom examples. Our indexing prompt is now automatically tuned to our new domain, as shown in Code Sample 3. 

• Code Sample 3: Auto-tuned prompt Goal
  Given a text document that is potentially relevant to this activity, first identify all entities needed from the text in order to capture the information and ideas in the text.
  Next, report all relationships among the identified entities.
  
  Steps
  
  1. Identify all entities. For each identified entity, extract the following information:
  • entity_name: Name of the entity, capitalized
  • entity_type: Suggest several labels or categories for the entity. The categories should not be specific, but should be as general as possible.
  • entity_description: Comprehensive description of the entity’s attributes and activities
  Format each entity as (““entity”“, , , )
  
  
  2. From the entities identified in step 1, identify all pairs of (source_entity, target_entity) that are *clearly related* to each other. For each pair of related entities, extract the following information:
  • source_entity: name of the source entity, as identified in step 1
  • target_entity: name of the target entity, as identified in step 1
  • relationship_description: explanation as to why you think the source entity and the target entity are related to each other
  • relationship_strength: a numeric score indicating strength of the relationship between the source entity and target entity
  Format each relationship as (““relationship”“, , , , )
  
  
  3. Return output in English as a single list of all the entities and relationships identified in steps 1 and 2. Use {record_delimiter} as the list delimiter.
  
  
  4. When finished, output
  Examples
  #############################
  
  Example 1:
  Text:
  CHRIS URMSON: Yeah, no, and it is, right? I think one of the things that people outside of Silicon Valley who haven’t been here don’t realize is that it’s not really. That, like, you know, people talk about Silicon Valley engineers being risk-takers. I think it’s actually the opposite. It’s the realization that if you go and try one of these things and you’re actually good at what you do, if it fails, it fails. You’ll have a job the next day at somewhere else, right? And you’ll have this wealth of experience that people will value. And I think that is something that it’s hard, you know, I’ll categorize this as you know east coast people but, you know, kind of more conventional business folks haven’t — don’t kind of have that sense of the opportunities that are around. And maybe we’ve just been here during a particularly
  
  Output:
  (“entity”, CHRIS URMSON, PERSON, Chris Urmson is a speaker discussing the culture and dynamics of Silicon Valley, particularly the attitude towards risk and failure)
  (“entity”, SILICON VALLEY, LOCATION, A region in California known for its technology industry and innovative environment, where engineers are perceived as risk-takers)
  (“entity”, SILICON VALLEY ENGINEERS, GROUP, Engineers working in Silicon Valley, characterized by a culture that values risk-taking and resilience in the face of failure)
  (“entity”, EAST COAST PEOPLE, GROUP, People from the East Coast of the United States, implied to have a more conventional and less risk-tolerant approach to business compared to those in Silicon Valley)
  
  (“relationship”, CHRIS URMSON – SILICON VALLEY, Chris Urmson discusses the culture and dynamics of Silicon Valley, emphasizing the local attitude towards risk and failure, 8)
  (“relationship”, SILICON VALLEY ENGINEERS – SILICON VALLEY, Silicon Valley Engineers are part of the Silicon Valley ecosystem, embodying the local culture of risk-taking and resilience, 9)
  (“relationship”, EAST COAST PEOPLE – SILICON VALLEY, East Coast People are contrasted with Silicon Valley individuals in terms of business culture and risk tolerance, 7)
  
  
  Example 2:
  Text:
  to ask Dr. Jemison that I think for her, and for me, space was this idea that really inspired us, I think, to go explore new frontiers. You know, it was this imagination of this thing that, you know, for me at least, like made me want to study computer science, because – like that was the most interesting terrestrial frontier I could go explore. And like you know, the thing that I wonder about is like what that frontier is, like what that inspiration will be for the next generation of scientists, and engineers and explorers. You know, like maybe it’s synthetic biology, but it’s going to be interesting to see whatever it is. [MUSIC] CHRISTINA WARREN: I couldn’t agree more. I look forward to watching and learning from all of that. All right, well, that’s a wrap. Thank you so much to Mae for joining us today. And to our listeners. Thank you for joining us and
  
  Output:
  (“entity”, SPACE, CONCEPT, Space is described as an inspiring concept that motivates exploration and study in new frontiers, particularly in science and technology)
  (“entity”, COMPUTER SCIENCE, FIELD, Computer science is highlighted as an interesting terrestrial frontier that the speaker was motivated to explore due to the inspiration from space)
  (“entity”, SYNTHETIC BIOLOGY, FIELD, Synthetic biology is mentioned as a potential inspiring frontier for the next generation of scientists, engineers, and explorers)
  (“entity”, CHRISTINA WARREN, PERSON, Christina Warren is the speaker who expresses agreement and looks forward to learning from the developments in new scientific frontiers)
  (“entity”, MAE, PERSON, Mae is mentioned as a guest who joined Christina Warren in the discussion about future scientific frontiers)
  
  (“relationship”, SPACE – COMPUTER SCIENCE, Space as a concept inspired the speaker to study computer science, 8)
  (“relationship”, CHRISTINA WARREN – MAE, Christina Warren thanks Mae for joining the discussion, 7)
  
  
  Example 3:
  Text:
  educational outcomes for kids. And if you look at the children of immigrants in East San Jose or East Palo Alto here in the Silicon Valley, like often, the parents are working two, three jobs. Like, they’re so busy that they have a hard time being engaged with their kids. And sometimes they don’t speak English. And so, like, they don’t even have the linguistic ability. And you can just imagine what a technology like this could do, where it really doesn’t care what language you speak. It can bridge that gap between the parents and the teacher, and it can be there to help the parent understand where the roadblocks are for the child and to even potentially get very personalized to the child’s needs and sort of help them on the things that they’re struggling with. I think it’s really, really very exciting. BILL GATES: Yeah, just the language barriers, we often forget about that. And that comes up in the developing world. India has
  
  Output:
  (“entity”, EAST SAN JOSE, GEO, A region in Silicon Valley where many immigrant families reside, and parents often work multiple jobs)
  (“entity”, EAST PALO ALTO, GEO, A region in Silicon Valley known for its significant immigrant population and economic challenges)
  (“entity”, SILICON VALLEY, GEO, A major hub for technology and innovation in California, USA)
  (“entity”, BILL GATES, PERSON, Prominent technology leader and philanthropist who discusses the impact of technology on overcoming language barriers)
  (“entity”, TECHNOLOGY, CONCEPT, Refers to new technological solutions that can assist in bridging language gaps between parents and teachers, and provide personalized support to children)
  
  (“relationship”, EAST SAN JOSE – SILICON VALLEY, East San Jose is a part of Silicon Valley, 9)
  (“relationship”, EAST PALO ALTO – SILICON VALLEY, East Palo Alto is located within Silicon Valley, 9)
  (“relationship”, TECHNOLOGY – BILL GATES, Bill Gates discusses the potential of technology to solve language barriers and educational challenges, 8)
  
  
  Real Data
  ######################
  
  Text: {input_text}
  
  Output:

Here, the automatically generated prompt using the sample content from Code Sample 2 identifies fourteen entity examples across six different entity types (person, location, group, concept, field, and geography) and eight relationship examples.

To assess how this impacts the extraction of the entire dataset, we used both the default and the auto-tuned prompt to generate the entity and relationship outputs. Before we explain the results, let’s review the default prompt’s outputs, which produced seven entities and six relationships, as shown in Code Sample 4. 

Code Sample 4: Default extraction output

Entities and Relationships:
(“entity”, ASHLEY LLORENS, PERSON, Ashley Llorens is a scientist, engineer, hip-hop artist, and vice president at Microsoft. He has worked at Johns Hopkins Applied Physics Laboratory and was nominated by the White House to serve as an AI expert for the Global Partnership on AI. He is also a voting member of the Recording Academy for the Grammy Awards.)
(“entity”, JOHNS HOPKINS APPLIED PHYSICS LABORATORY, ORGANIZATION, An institution where Ashley Llorens worked for two decades developing novel AI technologies and served as the founding chief of the intelligent systems center.)
(“entity”, WHITE HOUSE OFFICE OF SCIENCE AND TECHNOLOGY POLICY, ORGANIZATION, A U.S. government office that nominated Ashley Llorens to serve as an AI expert for the Global Partnership on AI.)
(“entity”, GLOBAL PARTNERSHIP ON AI, ORGANIZATION, An international initiative focused on AI where Ashley Llorens serves as an expert.)
(“entity”, RECORDING ACADEMY, ORGANIZATION, An organization responsible for the Grammy Awards, of which Ashley Llorens is a voting member.)
(“entity”, MICROSOFT, ORGANIZATION, A multinational technology company where Ashley Llorens joined as a vice president, distinguished scientist, and managing director for Microsoft Research.)
(“entity”, KEVIN SCOTT, PERSON, The host of the podcast where Ashley Llorens was interviewed and welcomed to Microsoft.)

(“relationship”, ASHLEY LLORENS – JOHNS HOPKINS APPLIED PHYSICS LABORATORY, Ashley Llorens worked at Johns Hopkins Applied Physics Laboratory for two decades, 9)
(“relationship”, ASHLEY LLORENS – WHITE HOUSE OFFICE OF SCIENCE AND TECHNOLOGY POLICY, Ashley Llorens was nominated by the White House Office of Science and Technology Policy to serve as an AI expert, 8)
(“relationship”, ASHLEY LLORENS – GLOBAL PARTNERSHIP ON AI, Ashley Llorens serves as an AI expert for the Global Partnership on AI, 8)
(“relationship”, ASHLEY LLORENS – RECORDING ACADEMY, Ashley Llorens is a voting member of the Recording Academy for the Grammy Awards, 7)
(“relationship”, ASHLEY LLORENS – MICROSOFT, Ashley Llorens joined Microsoft as a vice president and managing director for Microsoft Research, 9)
(“relationship”, KEVIN SCOTT – ASHLEY LLORENS, Kevin Scott interviewed Ashley Llorens on the podcast and welcomed him to Microsoft, 8)

Using the auto-tuned, domain-specific, automatically generated prompt, we achieved a deeper extraction, producing nine entities and eight relationships, as shown below in Code Sample 5. 

Code Sample 5: Auto-tuned extraction output

Entities and Relationships:
(“entity”, ASHLEY LLORENS, PERSON, Ashley Llorens is a scientist, engineer, hip-hop artist, and executive at Microsoft. He has a background in AI technology development and has been involved in the music industry as a hip-hop artist and Grammy voting member)
(“entity”, KEVIN SCOTT, PERSON, Kevin Scott is the host of the podcast where Ashley Llorens is a guest. He engages with Ashley about his background and career)
(“entity”, JOHNS HOPKINS APPLIED PHYSICS LABORATORY, ORGANIZATION, An institution where Ashley Llorens worked for two decades, developing novel AI technologies and leading the intelligent systems center)
(“entity”, WHITE HOUSE OFFICE OF SCIENCE AND TECHNOLOGY POLICY, ORGANIZATION, A governmental office that nominated Ashley Llorens to serve as an AI expert for the Global Partnership on AI)
(“entity”, GLOBAL PARTNERSHIP ON AI, ORGANIZATION, An international initiative focused on AI where Ashley Llorens serves as an expert.)
(“entity”, RECORDING ACADEMY, ORGANIZATION, An organization responsible for the Grammy Awards, of which Ashley Llorens is a voting member.)
(“entity”, MICROSOFT, ORGANIZATION, A major technology company where Ashley Llorens recently joined as a vice president, distinguished scientist, and managing director for Microsoft Research)
(“entity”, CHICAGO, LOCATION, The city where Ashley Llorens grew up, specifically mentioned as the south side and south suburbs, which influenced his interest in music and technology)
(“entity”, HIP-HOP, MUSIC GENRE, A music genre that significantly influenced Ashley Llorens during his childhood in Chicago, leading him to pursue a career in music alongside his technical career)

(“relationship”, ASHLEY LLORENS – JOHNS HOPKINS APPLIED PHYSICS LABORATORY, Ashley Llorens worked at Johns Hopkins Applied Physics Laboratory for two decades, developing AI technologies, 9)
(“relationship”, ASHLEY LLORENS – WHITE HOUSE OFFICE OF SCIENCE AND TECHNOLOGY POLICY, Ashley Llorens was nominated by the White House Office of Science and Technology Policy to serve as an AI expert, 9)
(“relationship”, ASHLEY LLORENS – GLOBAL PARTNERSHIP ON AI, Ashley Llorens serves as an AI expert for the Global Partnership on AI, 9)
(“relationship”, ASHLEY LLORENS – RECORDING ACADEMY, Ashley Llorens is a voting member of the Recording Academy for the Grammy Awards, 7)
(“relationship”, ASHLEY LLORENS – MICROSOFT, Ashley Llorens recently joined Microsoft as a vice president and managing director for Microsoft Research, 9)
(“relationship”, ASHLEY LLORENS – CHICAGO, Ashley Llorens grew up in Chicago, which influenced his early interest in music, particularly hip-hop, 7)
(“relationship”, ASHLEY LLORENS – HIP-HOP, Ashley Llorens was deeply influenced by hip-hop music during his upbringing in Chicago, leading him to pursue a career in music, 8)
(“relationship”, KEVIN SCOTT – ASHLEY LLORENS, Kevin Scott hosts Ashley Llorens on the podcast, discussing his background and career transitions, 7)

Compared with the default prompt, the auto-tuned prompt is an improvement, with more entities and more relationships, providing a more comprehensive view of our data. One key difference between this output and the output from the default prompt is the expansion in entity types being extracted. The default prompt is limited to three example types: organization, geography, and person. However, the auto-tuned prompt expands to more example types derived from the sample input text: organization, person, location, and music genre. 

Putting it all together 

We can observe a clear difference in the final outputs after using these auto-tuned prompts for indexing the podcast source data. To measure this difference, we compared the size of the resulting knowledge graphs using default with auto-tuned prompts. The following results were achieved while keeping all parameters constant between both runs and using GPT4-Turbo: 

 Entities Relationships Communities Default prompt 1796 2851 352 Auto-tuned prompt 4896 8210 1027 

As shown, the use of auto-tuning yields a significantly larger knowledge graph. For example, a prompt that looks for molecules will extract much more from a chemistry dataset than one that looks for people and places. More communities in the knowledge graph means that it can better serve global search queries. While the size of the knowledge graph can be a proxy for its utility, the true measure of utility is observed in end-to-end queries. To demonstrate this, we provide the following side-by-side comparison of a question using the default and auto-tuned prompts.

We asked GraphRAG the following question: “What’s the relationship between Ashley Llorens and Chicago?” 

First, we got the following results using the knowledge graph built with the default prompt. We then used the knowledge graph built with our auto-tuned prompt:

Default query results 

“I am sorry but I am unable to answer this question given the provided data.“

Auto-tuned query results 

“**Origin and Influence**: Ashley Llorens has a significant connection to Chicago, which has deeply influenced his personal and professional life. Growing up in a city known for its vibrant music scene, particularly hip-hop, has shaped his musical pursuits and artistic expression.“

“**Professional Impact**: His background and experiences in Chicago have carried over into his professional environment, notably at the Applied Physics Laboratory, where he has integrated elements of hip-hop culture [Data: Reports (940)].“

With auto-tuned indexing prompts, our knowledge graph became more representative of the dataset’s entities and relationships, enabling it to yield a valid response to the query. 

Advancing auto-tuning and expanding GraphRag’s capabilities

Looking ahead, we’re exploring other methods to build on this auto-tuning work. We’re excited to investigate new approaches for creating the core GraphRAG knowledge graph and are also studying ways to measure and evaluate the quality of these graph structures. Additionally, we’re researching methods to better assess performance so that we can identify the types of queries where GraphRAG provides unique value. This includes evaluating human-generated versus auto-tuned prompts, as well as exploring potential improvements to the auto-tuner. 

Overall, these new auto-tuner developments make GraphRAG much more accessible and turnkey. We hope this auto-tuning work removes many of the challenges involved when working with new datasets. We invite you to try out these capabilities yourself using GraphRAG’s core library (opens in new tab) and our Azure-based solution accelerator, available on GitHub (opens in new tab).

Try out GraphRAG Try out GraphRAG Accelerator Opens in a new tab

The post GraphRAG auto-tuning provides rapid adaptation to new domains appeared first on Microsoft Research.