Problem Solving

Large Language Models (LLMs) have attained human-level accuracy on medical question-answer (QA) benchmarks. However, their limitations in navigating clinical scenarios requiring flexible reasoning have recently been shown, raising concerns about the robustness and generalizability of LLM reasoning across diverse, real-world medical tasks. To probe potential LLM failure modes in clinical problem-solving, we present the medical abstraction and reasoning corpus (mARC-QA). mARC-QA assesses clinical reasoning through scenarios designed to exploit the Einstellung effect-the fixation of thought arising from prior experience, targeting LLM inductive biases toward inflexible pattern matching from their training data rather than engaging in flexible reasoning. We find that LLMs, including current state-of-the-art o1, Gemini, Claude, and DeepSeek models, perform poorly compared to physicians on mARC-QA, often demonstrating lack of commonsense medical reasoning and a propensity to hallucinate. In addition, uncertainty estimation analyses indicate that LLMs exhibit overconfidence in their answers, despite their limited accuracy. The failure modes revealed by mARC-QA in LLM medical reasoning underscore the need to exercise caution when deploying these models in clinical settings.

ObjectivesTo assess the effects of and related evidence certainty of interventions for attention deficit/hyperactivity disorder (ADHD) across an individual's lifespan, and to develop a continuously updated web platform for people with lived experience of ADHD as a method to disseminate living evidence synthesis for shared decision making.DesignUmbrella review and platform for shared decision making.Data sourcesSix databases from inception to 19 January 2025. Study authors were contacted for additional information when necessary.Eligibility criteria for selecting studiesSystematic reviews that used meta-analyses of randomised controlled trials were eligible if they compared a drug or non-drug intervention with a passive control in individuals with a diagnosis of ADHD. Primary outcomes were severity of ADHD symptoms, analysed by rater type (clinician-rated, parent-rated, teacher-rated, or self-rated) and time point (short term (12 weeks, or study endpoint), medium term (26 weeks), and long term (52 weeks)),acceptability (participants dropping out for any reason), and tolerability (participants dropping out owing to any side effects). Secondary outcomes included daily functioning, quality of life, comorbid symptoms, and key side effects (decreased sleep and appetite).Data synthesisEligible meta-analyses were re-estimated with a standardised statistical approach. Methodological quality was assessed using AMSTAR-2. Evidence certainty was evaluated using an algorithmic version of the GRADE framework, adapted for drug and non-drug interventions.Results115 of 414 full text articles were deemed eligible and 299 were excluded; the eligible articles comprised 221 unique combinations of participants, interventions, comparators, and outcomes. For each combination, the most recent and methodologically robust meta-analysis was selected for re-estimation, which gave 221 re-estimated meta-analyses in total, derived from 47 meta-analytic reports. In the short term, alpha-2 agonists, amphetamines, atomoxetine, methylphenidate, and viloxazine showed medium to large effect sizes in reducing the severity of ADHD symptoms in children and adolescents, with moderate to high certainty evidence. Methylphenidate showed consistent benefits across raters (standardised mean difference >0.75, 95% confidence interval (CI) 0.56 to 1.03; moderate or high certainty evidence). These interventions showed lower tolerability than the placebo, but this effect was not significant for methylphenidate and atomoxetine. In adults, atomoxetine, cognitive behavioural therapy, methylphenidate (and, when restricting analyses to high quality trials, amphetamines) showed at least moderate certainty evidence of efficacy on ADHD symptoms, with medium effect sizes. Methylphenidate, amphetamines, and atomoxetine had worse tolerability than placebo (methylphenidate, risk ratio 0.50, 95% CI 0.36 to 0.69; amphetamines, 0.40, 0.22 to 0.72; atomoxetine, 0.45, 0.35 to 0.58). Some non-drug interventions (acupuncture and cognitive behavioural therapy in children and adolescents, and mindfulness in adults) showed large effect sizes for ADHD symptoms, but with low certainty evidence. No high certainty, long term evidence was found for any intervention. An online platform showing effects and evidence certainty of each intervention across age groups, time points, and outcomes (https://ebiadhd-database.org/) was developed.ConclusionsThis review provides updated evidence to inform patients, practitioners, and guideline developers how best to manage ADHD symptoms. The online platform should facilitate the implementation of shared decision making in daily practice.Trial registrationOpen Science Framework https://osf.io/ugqy6/.

ImportanceShared decision-making is a widely promoted approach, yet clinicians, typically supportive in principle, find it difficult to implement because of concerns and barriers they commonly encounter in practice.ObjectiveTo generate a primer that describes shared decision-making from the perspective of clinicians.MethodsWe collaborated with clinicians, patient representatives, and health service researchers. We invited members of the International Society of Shared Decision Making to co-produce a primer for clinicians using a series of jointly edited online documents. We shared drafts with other clinicians and patients. Finally, we integrated the contributions until we had arrived at a consensus.FindingsTwenty-five people from 13 countries contributed; 9 had medical qualifications, 4 had nursing qualifications, and 12 others had a range of backgrounds. A total of 30 patients and clinicians provided further comments. The description differs from previous versions because it addresses the barriers that clinicians frequently mention. It describes how to overcome common challenges by emphasizing the importance of a clear invitation at initiation; it suggests how to manage patients' resistance to shouldering decisional responsibility; reinforces the need to allow time for deliberation, especially with other stakeholders; and reassures clinicians that consensus, albeit welcome, need not be the goal of shared decision-making.Conclusions and relevanceThis primer portrays a reflective clinician who is aware of power asymmetry, patient vulnerability, risk communication, health literacy, agenda setting, and goal clarification. It envisages a clinician who is curious about personal perspectives and who can offer collaborative, iterative, and deliberative steps.

The Allergic Rhinitis and its Impact on Asthma (ARIA) guidelines produced their first edition in 1999, with subsequent revisions in 2008, 2010, 2016 and 2019. A new iteration of ARIA-ARIA 2024-2025-in collaboration with EAACI is currently being developed, focusing on the management of allergic rhinitis. ARIA 2024-2025 follows the GRADE framework and is endorsed by the European Academy of Allergy and Clinical Immunology (EAACI). A set of approaches has been used to develop guideline questions, including surveying key opinion leaders and using artificial intelligence (AI)-based tools to analyse web searches on allergic rhinitis and to generate questions. Each prioritised guideline question is assessed through an Evidence-to-Decision (EtD) framework. EtDs support the systematic and transparent formulation of recommendations, comprising 12 criteria for which the best available evidence should be sought. In the context of ARIA-EAACI 2024-2025, such evidence is derived not only from randomised controlled trials but also-among others-from patient-generated data sources that better reflect the affected individuals' perspectives. Moreover, ARIA-EAACI 2024-2025 incorporates evidence on planetary health. Developed guideline recommendations will support the creation of digitalised decision algorithms and care pathways. This paper describes the methodology used to develop the person-centred, digitally enabled and AI-assisted ARIA-EAACI 2024-2025. Among others, it describes (i) the development and prioritisation of guideline questions, (ii) sources of evidence for EtDs and (iii) the development of digitalised decision algorithms and care pathways.