Paul A Nutting

Rutgers New Jersey Medical School, Newark, NJ, United States

Are you Paul A Nutting?

Claim your profile

Publications (111)464.96 Total impact

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Efforts to improve primary care diabetes management have assessed strategies across heterogeneous groups of patients and practices. However, there is substantial variability in how well practices implement interventions and achieve desired outcomes. To examine practice contextual features that moderate intervention effectiveness. Secondary analysis of data from a cluster randomized trial of three approaches for implementing the Chronic Care Model to improve diabetes care. Forty small to mid-sized primary care practices participated, with 522 clinician and staff member surveys. Outcomes were assessed for 822 established patients with a diagnosis of type 2 diabetes who had at least one visit to the practice in the 18 months following enrollment. The primary outcome was a composite measure of diabetes process of care, ascertained by chart audit, regarding nine quality measures from the American Diabetes Association Physician Recognition Program: HgA1c, foot exam, blood pressure, dilated eye exam, cholesterol, nephropathy screen, flu shot, nutrition counseling, and self-management support. Data from practices included structural and demographic characteristics and Practice Culture Assessment survey subscales (Change Culture, Work Culture, Chaos). Across the three implementation approaches, demographic/structural characteristics (rural vs. urban + .70(p = .006), +2.44(p < .001), -.75(p = .004)); Medicaid: <20 % vs. ≥20 % (-.20(p = .48), +.75 (p = .08), +.60(p = .02)); practice size: <4 clinicians vs. ≥4 clinicians (+.56(p = .02), +1.96( p < .001), +.02(p = .91)); practice Change Culture (high vs. low: -.86(p = .048), +1.71(p = .005), +.34(p = .22)), Work Culture (high vs. low: -.67(p = .18), +2.41(p < .001), +.67(p = .005)) and variability in practice Change Culture (high vs. low: -.24(p = .006), -.20(p = .0771), -.44(p = .0019) and Work Culture (high vs. low: +.56(p = .3160), -1.0(p = .008), -.25 (p = .0216) were associated with trajectories of change in diabetes process of care, either directly or differentially by study arm. This study supports the need for broader use of methodological approaches to better examine contextual effects on implementation and effectiveness of quality improvement interventions in primary care settings.
    Journal of general internal medicine. 12/2014;
  • [Show abstract] [Hide abstract]
    ABSTRACT: Current research on primary care practice redesign suggests that outside facilitation can be an important source of support for achieving substantial change.
    Family practice. 10/2014;
  • [Show abstract] [Hide abstract]
    ABSTRACT: Metrics focus attention on what is important. Balanced metrics of primary health care inform purpose and aspiration as well as performance. Purpose in primary health care is about improving the health of people and populations in their community contexts. It is informed by metrics that include long-term, meaning- and relationship-focused perspectives. Aspirational uses of metrics inspire evolving insights and iterative improvement, using a collaborative, developmental perspective. Performance metrics assess the complex interactions among primary care tenets of accessibility, a whole-person focus, integration and coordination of care, and ongoing relationships with individuals, families, and communities; primary health care principles of inclusion and equity, a focus on people's needs, multilevel integration of health, collaborative policy dialogue, and stakeholder participation; basic and goal-directed health care, prioritization, development, and multilevel health outcomes. Environments that support reflection, development, and collaborative action are necessary for metrics to advance health and minimize unintended consequences.
    Annual Review of Public Health 03/2014; 35:423-442. · 3.27 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Improving the patient experience of primary care is a stated focus of efforts to transform primary care practices into "Patient-centered Medical Homes" (PCMH) in the United States, yet understanding and promoting what defines a positive experience from the patient's perspective has been de-emphasized relative to the development of technological and communication infrastructure at the PCMH. The objective of this qualitative study was to compare primary care clinicians' and their patients' perceptions of the patients' experiences, expectations and preferences as they try to achieve care for depression. We interviewed 6 primary care clinicians along with 30 of their patients with a history of depressive disorder attending 4 small to medium-sized primary care practices from rural and urban settings. Three processes on the way to satisfactory depression care emerged: 1. a journey, often from fractured to connected care; 2. a search for a personal understanding of their depression; 3. creation of unique therapeutic spaces for treating current depression and preventing future episodes. Relative to patients' observations regarding stigma's effects on accepting a depression diagnosis and seeking treatment, clinicians tended to underestimate the presence and effects of stigma. Patients preferred clinicians who were empathetic listeners, while clinicians worried that discussing depression could open "Pandora's box" of lengthy discussions and set them irrecoverably behind in their clinic schedules. Clinicians and patients agreed that somatic manifestations of mental distress impeded the patients' ability to understand their suffering as depression. Clinicians reported supporting several treatment modalities beyond guideline-based approaches for depression, yet also displayed surface-level understanding of the often multifaceted support webs their patient described. Improving processes and outcomes in primary care may demand heightened ability to understand and measure the patients' experiences, expectations and preferences as they receive primary care. Future research would investigate a potential mismatch between clinicians' and patients' perceptions of the effects of stigma on achieving care for depression, and on whether time spent discussing depression during the clinical visit improves outcomes. Improving care and outcomes for chronic disorders such as depression may require primary care clinicians to understand and support their patients' unique 'therapeutic spaces.'
    BMC Family Practice 01/2014; 15(1):13. · 1.61 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: The application of complexity science to understanding healthcare system improvement highlights the need to consider interdependencies within the system. One important aspect of the interdependencies in healthcare delivery systems is how individuals relate to each other. However, results from our observational and interventional studies focusing on relationships to understand and improve outcomes in a variety of healthcare settings have been inconsistent. We sought to better understand and explain these inconsistencies by analyzing our findings across studies and building new theory. We analyzed eight observational and interventional studies in which our author team was involved as the basis of our analysis, using a set theoretical qualitative comparative analytic approach. Over 16 investigative meetings spanning 11 months, we iteratively analyzed our studies, identifying patterns of characteristics that could explain our set of results.Our initial focus on differences in setting did not explain our mixed results. We then turned to differences in patient care activities and tasks being studied and the attributes of the disease being treated. Finally, we examined the interdependence between task and disease. We identified system-level uncertainty as a defining characteristic of complex systems through which we interpreted our results. We identified several characteristics of healthcare tasks and diseases that impact the ways uncertainty is manifest across diverse care delivery activities. These include disease-related uncertainty (pace of evolution of disease and patient control over outcomes) and task-related uncertainty (standardized versus customized, routine versus non-routine, and interdependencies required for task completion). Uncertainty is an important aspect of clinical systems that must be considered in designing approaches to improve healthcare system function. The uncertainty inherent in tasks and diseases, and how they come together in specific clinical settings, will influence the type of improvement strategies that are most likely to be successful. Process-based efforts appear best-suited for low-uncertainty contexts, while relationship-based approaches may be most effective for high-uncertainty situations.
    Implementation Science 01/2014; 9(1):165. · 2.37 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Innovative workforce models are being developed and implemented to meet the changing demands of primary care. A literature review was conducted to construct a typology of workforce models used by primary care practices. Ovid Medline, CINAHL, and PsycInfo were used to identify published descriptions of the primary care workforce that deviated from what would be expected in the typical practice in the year 2000. Expert consultants identified additional articles that would not show up in a regular computerized search. Full texts of relevant articles were read and matrices for sorting articles were developed. Each article was reviewed and assigned to one of 18 cells in the matrices. Articles within each cell were then read again to identify patterns and develop an understanding of the full spectrum of workforce innovation within each category. This synthesis led to the development of a typology of workforce innovations represented in the literature. Many workforce innovations added personnel to existing practices, whereas others sought to retrain existing personnel or even develop roles outside the traditional practice. Most of these sought to minimize the impact on the existing practice roles and functions, particularly that of physicians. The synthesis also identified recent innovations which attempted to fundamentally transform the existing practice, with transformation being defined as a change in practice members' governing variables or values in regard to their workforce role. Most conceptualizations of the primary care workforce described in the literature do not reflect the level of innovation needed to meet the needs of the burgeoning numbers of patients with complex health issues, the necessity for roles and identities of physicians to change, and the call for fundamentally redesigned practices. However, we identified 5 key workforce innovation concepts that emerged from the literature: team care, population focus, additional resource support, creating workforce connections, and role change.
    Medical care 12/2013; · 3.24 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Transforming small independent practices to patient-centered medical homes is widely believed to be a critical step in reforming the US health care system. Our team has conducted research on improving primary care practices for more than fifteen years. We have found four characteristics of small primary care practices that seriously inhibit their ability to make the transformation to this new care model. We found that small practices were extremely physician-centric, lacked meaningful communication among physicians, were dominated by authoritarian leadership behavior, and were underserved by midlevel clinicians who had been cast into unimaginative roles. Our analysis suggests that in addition to payment reform, a shift in the mind-set of primary care physicians is needed. Unless primary care physicians can adopt new mental models and think in new ways about themselves and their practices, it will be very difficult for them and their practices to create innovative care teams, become learning organizations, and act as good citizens within the health care neighborhood.
    Health Affairs 11/2012; 31(11):2417-22. · 4.64 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Change champions are important for moving new innovations through the phases of initiation, development, and implementation. Although research attributes positive health care changes to the help of champions, little work provides details about the champion role. Using a combination of immersion/crystallization and matrix techniques, we analyzed qualitative data, which included field notes of team meetings, interviews, and transcripts of facilitator meetings, from a sample of 8 practices. Our analysis yielded insights into the value of having 2 discrete types of change champions: (1) those associated with a specific project (project champions) and (2) those leading change for entire organizations (organizational change champions). Relative to other practices under study, those that had both types of champions who complemented each other were best able to implement and sustain diabetes care processes. We provide insights into the emergence and development of these champion types, as well as key qualities necessary for effective championing. Practice transformation requires a sustained improvement effort that is guided by a larger vision and commitment and assures that individual changes fit together into a meaningful whole. Change champions-both project and organizational change champions-are critical players in supporting both innovation-specific and transformative change efforts.
    The Journal of the American Board of Family Medicine 09/2012; 25(5):676-85. · 1.76 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Quality improvement collaboratives (QICs) are used extensively to promote quality improvement in health care. Evidence of their effectiveness is limited, prompting calls to "open up the black box" to better understand how and why such collaboratives work. We selected a cohort of 5 primary care practices that participated in a 6-month intervention study aimed at improving colorectal cancer screening rates. Using an immersion/crystallization technique, we analyzed qualitative data that included audio recordings and field notes of QICs and practice-based team meetings. Three themes emerged from our analysis: (1) practice staff became empowered through and drew on the QICs to advance change efforts in the face of leader/physician resistance; (2) a mix of content and media in the QIC program was important for reaching all participants; (3) resources offered at the QIC did little to spur practice change efforts. QICs offer a potentially powerful way of disseminating health care innovations through enhanced strategies for learning and change. Creating collaborative environments in which diverse participants learn, listen, reflect, and share together can enable them to take back to their own organizations key messages and change strategies that benefit them the most.
    The Journal of the American Board of Family Medicine 03/2012; 25(2):149-57. · 1.76 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The authors describe the implementation of a depression care management (DCM) program at Colorado Access, a public sector health plan, and describe the program's clinical and system outcomes for members with chronic medical conditions. High medical risk, high cost Medicaid health plan members were identified and systematically screened for depression. A total of 370 members enrolled in the DCM program. Longitudinal analyses revealed significantly reduced depression severity scores at 3, 6, and 12 months after intervention as compared to baseline depression scores. At 12 months, 56% of enrollees in the DCM program had either a 50% reduction in PHQ-9 scores or a PHQ-9 score < 10. Longitudinal economic analyses comparing 12 months before and after intervention revealed a significant but modest increase in ER visits, outpatient office visits, and overall medical and pharmacy costs when adjusted for months enrolled in DCM. Limitations and recommendations for the integrated depression care management are discussed.
    Depression research and treatment 01/2012; 2012:769298.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Serious shortcomings remain in clinical care in the United States despite widespread use of improvement strategies for enhancing clinical performance based on knowledge transfer approaches. Recent calls to transform primary care practice to a patient-centered medical home present even greater challenges and require more effective approaches. Our research team conducted a series of National Institutes of Health funded descriptive and intervention projects to understand organizational change in primary care practice settings, emphasizing a complexity science perspective. The result was a developmental research effort that enabled the identification of critical lessons relevant to enabling practice change. A summary of findings from a 15-year program of research highlights the limitations of viewing primary care practices in the mechanistic terms that underlie current or traditional approaches to quality improvement. A theoretical perspective that views primary care practices as dynamic complex adaptive systems with "agents" who have the capacity to learn, and the freedom to act in unpredictable ways provides a better framework for grounding quality improvement strategies. This framework strongly emphasizes that quality improvement interventions should not only use a complexity systems perspective, but also there is a need for continual reflection, careful tailoring of interventions, and ongoing attention to the quality of interactions among agents in the practice. It is unlikely that current strategies for quality improvement will be successful in transforming current primary care practice to a patient-centered medical home without a stronger guiding theoretical foundation. Our work suggests that a theoretical framework guided by complexity science can help in the development of quality improvement strategies that will more effectively facilitate practice change.
    Medical care 12/2011; 49 Suppl:S28-35. · 3.24 Impact Factor
  • Paul A Nutting, Carlos R Jaén
    Health Affairs 06/2011; 30(6):1216. · 4.64 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Many commentators view the conversion of small, independent primary care practices into patient-centered medical homes as a vital step in creating a better-performing health care system. The country's first national medical home demonstration, which ran from June 1, 2006, to May 31, 2008, and involved thirty-six practices, showed that this transformation can be lengthy and complex. Among other features, the transformation process requires an internal capability for organizational learning and development; changes in the way primary care clinicians think about themselves and their relationships with patients as well as other clinicians on the care team; and awareness on the part of primary care clinicians that they will need to make long-term commitments to change that may require three to five years of external assistance. Additionally, transforming primary care requires synchronizing practice redesign with development of the health care "neighborhood," which is made up of a broad range of health and health care resources available to patients. It also requires payment reform that supports practice development and a policy environment that sets reasonable expectations and time frames for the adoption of appropriate innovations.
    Health Affairs 03/2011; 30(3):439-45. · 4.64 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The patient-centered medical home (PCMH) has become a widely cited solution to the deficiencies in primary care delivery in the United States. To achieve the magnitude of change being called for in primary care, quality improvement interventions must focus on whole-system redesign, and not just isolated parts of medical practices. Investigators participating in 9 different evaluations of Patient Centered Medical Home implementation shared experiences, methodological strategies, and evaluation challenges for evaluating primary care practice redesign. A year-long iterative process of sharing and reflecting on experiences produced consensus on 7 recommendations for future PCMH evaluations: (1) look critically at models being implemented and identify aspects requiring modification; (2) include embedded qualitative and quantitative data collection to detail the implementation process; (3) capture details concerning how different PCMH components interact with one another over time; (4) understand and describe how and why physician and staff roles do, or do not evolve; (5) identify the effectiveness of individual PCMH components and how they are used; (6) capture how primary care practices interface with other entities such as specialists, hospitals, and referral services; and (7) measure resources required for initiating and sustaining innovations. Broad-based longitudinal, mixed-methods designs that provide for shared learning among practice participants, program implementers, and evaluators are necessary to evaluate the novelty and promise of the PCMH model. All PCMH evaluations should as comprehensive as possible, and at a minimum should include a combination of brief observations and targeted qualitative interviews along with quantitative measures.
    Medical care 11/2010; 49(1):10-6. · 3.24 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: This article introduces a journal supplement evaluating the country's first national demonstration of the patient-centered medical home (PCMH) concept. The PCMH is touted by some as a linchpin for renewing the foundering US health care system and its primary care foundation. The National Demonstration Project (NDP) tested a new model of care and compared facilitated and self-directed implementation approaches in a group-randomized clinical trial. The NDP asked what a national sample of 36 highly motivated family practices could accomplish in moving toward the PCMH ideal during 2 years within the current US health care payment and organizational system. Our independent evaluation used a multimethod approach that integrated qualitative methods to tell the NDP story from multiple perspectives and quantitative methods to assess and compare aspects that could be measured. The 7 scientific reports presented in this supplement explain the process, outcomes, lessons, and implications of the NDP. This introductory article provides context for making sense of the NDP. Important context includes the evolution of the PCMH concept and movement, the roots of the NDP and how it developed, and both what is valuable and what is problematic about family medicine and primary care. Together, the articles in this supplement show how primary care practices and the concept of the PCMH can continue to evolve. The evaluation depicts some of the early effects of this evolution on patients and practices, and shows how the process of practice development can be understood and how lessons from the NDP can inform ongoing and future efforts to transform primary care and health care systems.
    The Annals of Family Medicine 06/2010; 8 Suppl 1:S2-8; S92. · 4.61 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The patient-centered medical home (PCMH) is four things: 1) the fundamental tenets of primary care: first contact access, comprehensiveness, integration/coordination, and relationships involving sustained partnership; 2) new ways of organizing practice; 3) development of practices' internal capabilities, and 4) related health care system and reimbursement changes. All of these are focused on improving the health of whole people, families, communities and populations, and on increasing the value of healthcare. The value of the fundamental tenets of primary care is well established. This value includes higher health care quality, better whole-person and population health, lower cost and reduced inequalities compared to healthcare systems not based on primary care. The needed practice organizational and health care system change aspects of the PCMH are still evolving in highly related ways. The PCMH will continue to evolve as evidence comes in from hundreds of demonstrations and experiments ongoing around the country, and as the local and larger healthcare systems change. Measuring the PCMH involves the following: Giving primacy to the core tenets of primary care. Assessing practice and system changes that are hypothesized to provide added value Assessing development of practices' core processes and adaptive reserve. Assessing integration with more functional healthcare system and community resources. Evaluating the potential for unintended negative consequences from valuing the more easily measured instrumental features of the PCMH over the fundamental relationship and whole system aspects. Recognizing that since a fundamental benefit of primary care is its adaptability to diverse people, populations and systems, functional PCMHs will look different in different settings. Efforts to transform practice to patient-centered medical homes must recognize, assess and value the fundamental features of primary care that provide personalized, equitable health care and foster individual and population health.
    Journal of General Internal Medicine 06/2010; 25(6):601-12. · 3.28 Impact Factor
  • Source
    Family practice management 03/2010; 17(2):31-4.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: This article summarizes findings from the National Demonstration Project (NDP) and makes recommendations for policy makers and those implementing patient-centered medical homes (PCMHs) based on these findings and an understanding of diverse efforts to transform primary care. The NDP was launched in June 2006 as the first national test of a particular PCMH model in a diverse sample of 36 family practices, randomized to facilitated or self-directed groups. An independent evaluation team used a multimethod evaluation strategy, analyzing data from direct observation, depth interviews, e-mail streams, medical record audits, and patient and clinical staff surveys. Peer-reviewed manuscripts from the NDP provide answers to 4 key questions: (1) Can the NDP model be built? (2) What does it take to build the NDP model? (3) Does the NDP model make a difference in quality of care? and (4) Can the NDP model be widely disseminated? We find that although it is feasible to transform independent practices into the NDP conceptualization of a PCMH, this transformation requires tremendous effort and motivation, and benefits from external support. Most practices will need additional resources for this magnitude of transformation. Recommendations focus on the need for the PCMH model to continue to evolve, for delivery system reform, and for sufficient resources for implementing personal and practice development plans. In the meantime, we find that much can be done before larger health system reform.
    The Annals of Family Medicine 01/2010; 8 Suppl 1:S80-90; S92. · 4.61 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The Using Learning Teams for Reflective Adaptation (ULTRA) study used facilitated reflective adaptive process (RAP) teams to enhance communication and decision making in hopes of improving adherence to multiple clinical guidelines; however, the study failed to show significant clinical improvements. The purpose of this study was to examine qualitative data from 25 intervention practices to understand how they engaged in a team-based collaborative change management strategy and the types of issues they addressed. We analyzed field notes and interviews from a multimethod practice assessment, as well as field notes and audio-taped recordings from RAP meetings, using an iterative group process and an immersion-crystallization approach. Despite a history of not meeting regularly, 18 of 25 practices successfully convened improvement teams. There was evidence of improved practice-wide communication in 12 of these practices. At follow-up, 8 practices continued RAP meetings and found the process valuable in problem solving and decision making. Seven practices failed to engage in RAP primarily because of key leaders dominating the meeting agenda or staff members hesitating to speak up in meetings. Although the number of improvement targets varied considerably, most RAP teams targeted patient care-related issues or practice-level organizational improvement issues. Not a single practice focused on adherence to clinical care guidelines. Primary care practices can successfully engage in facilitated team meetings; however, leaders must be engaged in the process. Additional strategies are needed to engage practice leaders, particularly physicians, and to target issues related to guideline adherence.
    The Annals of Family Medicine 01/2010; 8(5):425-32. · 4.61 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: We provide an overall description of the National Demonstration Project (NDP) intervention to transform family practices into patient-centered medical homes. An independent evaluation team used multiple data sources and methods to describe the design and implementation of the NDP. These included direct observation of the implementation team and project meetings, site visits to practices, depth interviews with practice members and implementation team members, access to practice communications (eg, telephone calls, e-mails), and public domain materials (eg, the NDP Web site). The American Academy of Family Physicians created a new division called TransforMED, which launched the 24-month NDP in June 2006. From 337 family medicine practices completing an extensive online application, 36 were selected and randomized to a facilitated group, which received tailored, intensive assistance and services from TransforMED, or a self-directed group, which received very limited assistance. Three facilitators from diverse backgrounds in finance, practice management, and organizational psychology used multiple practice change strategies including site visits, e-mails, metrics, and learning sessions. The self-directed practices worked primarily on their own, but self-organized a retreat midway through the project. The intervention model for the project evolved to be consistent with the emerging national consensus principles of the patient-centered medical home. The independent evaluation team studied the NDP and provided ongoing feedback to inform the implementation process. The NDP illustrates that complex practice change interventions must combine flexibility in the intervention model, implementation strategy, and the evaluation, in order to maximize ongoing learning.
    The Annals of Family Medicine 01/2010; 8 Suppl 1:S21-32; S92. · 4.61 Impact Factor

Publication Stats

4k Citations
464.96 Total Impact Points

Institutions

  • 2012
    • Rutgers New Jersey Medical School
      • Department of Family Medicine and Community Health (RWJ Medical School)
      Newark, NJ, United States
  • 1994–2011
    • University of Colorado
      • • The Research Center
      • • Department of Family Medicine
      Denver, CO, United States
  • 2010
    • University of Texas Health Science Center at San Antonio
      • Department of Family & Community Medicine
      San Antonio, TX, United States
    • University of North Texas at Dallas
      Dallas, Texas, United States
  • 2006–2010
    • Robert Wood Johnson University Hospital
      New Brunswick, New Jersey, United States
  • 2009
    • University of Texas at Austin
      • Department of Educational Psychology
      Texas City, TX, United States
  • 2002–2007
    • National Research Center (CO, USA)
      Boulder, Colorado, United States
  • 2005
    • Kaiser Permanente
      • Center for Health Research (Oregon, Hawaii, and Georgia)
      Oakland, California, United States
  • 2004
    • Geisel School of Medicine at Dartmouth
      Hanover, New Hampshire, United States
  • 2000
    • University of Missouri
      Columbia, Missouri, United States
  • 1997
    • Highland Hospital
      Oakland, California, United States
  • 1994–1997
    • University at Buffalo, The State University of New York
      • Department of Family Medicine
      Buffalo, NY, United States
  • 1995
    • Maine Medical Center
      Portland, Maine, United States
  • 1992
    • University of Rochester
      • Department of Family Medicine
      Rochester, NY, United States