SummarySpecialists and primary care physicians at San Francisco General Hospital use electronic communications to facilitate more rapid and efficient access to specialty expertise for patients served by the hospital's referral network of safety net clinics, many of whom are uninsured or underinsured. Using the "eReferral" system, primary care physicians and specialists exchange free text messages through a referral program embedded in the patient's electronic medical record; communication continues until the clinicians either schedule the patient for a specialist appointment or agree to a plan for managing the patient's condition in the primary care medical home. The program significantly reduced patient wait times for specialty appointments, improved communication between referring clinicians and specialists, and enhanced primary care physician perceptions of the quality of care provided in safety net clinics.Moderate: The evidence consists of pre- and post-implementation comparisons of wait times for patients referred for routine gastroenterology appointments, survey data from frontline specialists in 9 different clinics that implemented eReferral, survey data from 298 primary care providers in 24 clinics that use the system, and interviews with staff from 4 primary care clinics and 3 specialty services that use eReferral.
Developing OrganizationsSan Francisco General Hospital; University of California, San Francisco
Date First Implemented2005
The program began as a pilot in one specialty clinic in 2005, expanded to other specialty clinics beginning in 2007, and as of September 2013 is being used in more than 40 such clinics.
Vulnerable Populations > Impoverished; Insurance Status > Medicaid; Vulnerable Populations > Medically uninsured; Insurance Status > Uninsured
Problem AddressedUninsured and underinsured patients face significant barriers to accessing specialty care. Not enough specialists will accept referrals from safety net providers, and those who do often rely on inefficient referral systems, leading to long wait times, incomplete information, and suboptimal provider-patient interactions.
- Limited access to specialists, leading to delays in care: An insufficient number of specialists accept Medicaid and other forms of public insurance,1,2 leaving uninsured and underinsured patients with a choice of relatively few specialists. As a result, these patients face long wait times for appointments,3 which, in turn, can lead to diagnostic delays, higher costs, lost information, and poor health outcomes.4,5 For example, wait times in 2005 at San Francisco General Hospital's gastroenterology clinic had climbed to more than 11 months; by the time patients showed up for their appointments, the referral information was frequently unavailable, and in some cases the patient's condition had worsened. The first visit often consisted of ordering laboratory and radiology tests that ideally would have been obtained before the visit.
- Inefficient referral process that exacerbates the problem: Referring providers often use handwritten or faxed referrals, leading to poor or inadequate communication with specialists, including incomplete information and unclear consultative questions.1,6 Before the implementation of the eReferral system, specialty clinics at San Francisco General Hospital scheduled patients on a first-referred, first-scheduled basis without regard to clinical urgency. Referring providers who wished to expedite a patient's specialty appointment had to make an extra effort to personally reach a specialist by pager, telephone, or e-mail. In addition, specialists often had to spend a patient's entire first visit simply determining the nature of the complaint, a problem made even more challenging by the high percentage of safety net patients with limited English proficiency and low health literacy.
Description of the Innovative ActivitySpecialists and primary care physicians at San Francisco General Hospital use electronic communications to facilitate more rapid and efficient access to specialty expertise and consultations for patients served by the hospital's referral network of safety net clinics, many of whom are uninsured or underinsured. Using the eReferral system, primary care physicians and specialists exchange free text messages through a referral program embedded in the patient's electronic medical record (EMR); communication continues until the clinicians schedule the patient for a specialist appointment or agree to a plan for managing the patient’s condition in the primary care medical home. Key program elements include:
- Accessing the system: Primary care providers working in San Francisco General Hospital's referral network of safety net clinics (which includes 4 hospital-based clinics, 11 county-funded clinics, and 11 independent community health centers) can refer patients electronically to any of more than 40 hospital-based specialty clinics and services that have implemented the eReferral system. To refer a patient, the primary care provider accesses a Web-based program from within the patient's EMR. At many primary care clinics, providers can do this at computer terminals located within each patient care room; at the independent community health centers, providers usually have access to a more limited number of workspaces with EMR connectivity. The referring provider submits an electronic referral that includes a free text description of the reason for the referral, with relevant clinical, demographic, and provider information from the EMR automatically appended to the request. The provider submits this referral to the appropriate specialty clinic through the eReferral system. Use of the system is mandatory with any specialty clinic that has implemented it (i.e., referring clinics can no longer submit handwritten or faxed referrals to these clinics).
- Iterative communication to review, resolve case: Each specialty clinic designates one or more clinicians (either physicians or nurse practitioners) to serve as specialty reviewers. These individuals assess each referral submitted through the system for appropriateness, completeness, and urgency, and use the system to either approve the patient's appointment or initiate further discussions with the referring provider. Specialty reviewers respond to any referral submitted through the system within 3 business days. The response can follow one of several paths based on the specifics of the referral, as outlined below:
- Scheduling of approved appointments according to urgency: Those cases approved for an appointment become part of an electronic list of patients to be scheduled. For those in need of expedited care, the reviewer flags the referral as urgent and provides specific instructions to the clerical staff on when to schedule the appointment. A scheduler manually enters the appointment into the hospital's scheduling system. The hospital's EMR automatically generates an appointment notification letter for the patient and sends an e-mail to alert the referring provider about the appointment.
- Request for additional information: Specialty reviewers use the eReferral program to request additional diagnostic evaluation or information from the referring provider if the consultative request seems unclear, the information provided appears inadequate, or the diagnostic evaluation is incomplete. In these cases, referring providers receive an automated e-mail notifying them of the specialty reviewer's response. The referring provider then provides clarification or additional information through the system, with all communication being captured in real time within the patient's EMR. Based on this information, the specialist reviewer then determines if the patient should be scheduled for a visit.
- Providing consultation and guidance: In some cases, the patient can be managed by the referring provider with specialist guidance. In these instances, the reviewer uses the system to provide recommendations and to answer any consultative questions the referring provider has posed. When appropriate, the specialist reviewer can attach Web links or articles containing up-to-date medical information relevant to the patient's case.
- Direct scheduling for subset of services: For a set of 36 services where a qualified specialist reviewer is not available or a review is not necessary, referral requests get routed directly to a scheduler who makes the appointment electronically. Examples include audiology, electroencephalography, echocardiography, and optometry.
- Ratings project: In June 2011, a bidirectional rating system was implemented within the eReferral program to assess and improve the quality of communication between specialty reviewers and primary care providers. This project aimed to provide feedback and identify areas of improvement for both reviewers and referring providers. Specialty reviewers completed electronic surveys each month on a random sample of new eReferrals, rating the referring provider on the clarity of consultation, the appropriateness of the prereferral evaluation/workup, and the quality of the patient’s history provided. Additionally, the survey contained two optional checkbox questions for the specialist to indicate whether (1) the referring provider should have been able to manage the patient’s care without specialty guidance, and (2) the patient should have been managed more appropriately via a page to the specialty on-call fellow (e.g., urgent patient safety issue). Referring providers, in turn, completed a survey for every initially “not scheduled” eReferral response, rating the specialty reviewer on his/her helpfulness, the educational value of the response, and the extent to which he/she agreed with the specialist’s decision to “not schedule” the patient into clinic. The eReferral team has used the results from the surveys to provide feedback to both specialist reviewers and referring providers, with documented improvements in specialist ratings after the feedback sessions (manuscript in progress).
References/Related Articles(added September 2013) Chen AH, Murphy EJ, Yee HF Jr. eReferral — A new model for integrated care. N Engl J Med. June 2013;368:2450-2453. Available at:
Bindman AB, Chen A, Fraser JS, et al. Healthcare reform with a safety net: lessons from San Francisco. Am J Manag Care. 2009 Oct;15(10):747-50. [PubMed]
Straus SG, Chen AH, Yee H, et al. Implementation of an electronic referral system for outpatient specialty care. AMIA Annu Symp Proc. 2011;2011:1337-46. [PubMed]
Chen AH, Yee HF Jr. Improving primary care–specialty care communication: lessons from San Francisco's safety net. Arch Intern Med. 2011 Jan 10;171(1):65-7. [PubMed]
Chen AH, Kushel MB, Grumbach K, et al. Practice profile: a safety-net system gains efficiencies through ‘eReferrals’ to specialists. Health Aff (Millwood). 2010;29(5):969-71. [PubMed] Available at: http://content.healthaffairs.org/cgi/reprint/29/5/969.pdf (If you don't have the software to open this PDF, download free Adobe Acrobat Reader® software .).
Chen AH, Yee HF Jr. Improving the primary care–specialty care interface: getting from here to there. Arch Intern Med. 2009;169(11):1024-6. [PubMed]
Kim-Hwang JE, Chen AH, Bell D, et al. Evaluating electronic referrals for specialty care at a public hospital. J Gen Intern Med. ePub 2010 May 29. [PubMed]
Kim Y, Chen AH, Keith E, et al. Not perfect, but better: primary care providers' experiences with electronic referrals in the safety net health system. J Gen Int Med. 2009;24(5):614-9. [PubMed]
Contact the InnovatorAlice Hm Chen, MD, MPH
Chief Integration Officer
Director, eReferral Medical Services
University of California San Francisco/San Francisco General Hospital
San Francisco, CA 94143
Phone: (415) 206-4049
Innovator DisclosuresDr. Chen reported having no financial interests or business/professional affiliations relevant to the work described in this profile; however, information on funders is available in the Funding Sources section.
ResultsThe program significantly reduced patient wait times for specialty appointments, improved communication between referring clinicians and specialists, enhanced primary care provider perceptions of the quality of care offered by safety net clinics, and provided educational benefits for the referring provider.
Moderate: The evidence consists of pre- and post-implementation comparisons of wait times for patients referred for routine gastroenterology appointments, survey data from frontline specialists in 9 different clinics that implemented eReferral, survey data from 298 primary care providers in 24 clinics that use the system, and interviews with staff from 4 primary care clinics and 3 specialty services that use eReferral.
- Significantly shorter wait times: A 1-year pilot conducted in the hospital's gastroenterology clinic found that wait times for appointments fell from 11 months to 4 months after implementation of the system. (The clinic did not add capacity during this time period.) In the first nine clinics to adopt eReferral, the average wait time for an initial consultative visit was also reduced by nearly 60 percent.7
- Enhanced communication from referring clinicians: A survey of frontline specialists in nine different clinics found that, compared with the prior paper and fax-based system, use of eReferral resulted in improved clarity of communication. The percentage of instances in which it was difficult to identify the reason for the referral fell by nearly 50 percent in medical specialty clinics and nearly 75 percent in surgical specialty clinics.8
- Positive perceptions of quality from referring physicians: A survey of primary care providers who use the system found that 71.9 percent believe it has improved the quality of care provided to patients in safety net clinics. The same percentage believe that the system has enhanced the quality of guidance provided during the prespecialty visit workup. In addition, 57.3 percent of primary care physicians believe that the system has enhanced specialists' ability to address their clinical questions.1
- Educational benefits for referring clinicians: A qualitative analysis of key informant interviews found that the eReferral system provided opportunities for the specialist to give guidance, education, and co-management support for the referring provider.9
Context of the InnovationSan Francisco General Hospital, through a partnership with the University of California at San Francisco, serves as the primary provider of specialty care for the city's uninsured and underinsured patients. The city's sole public hospital, it is part of the San Francisco Department of Health, which also includes a network of safety net primary care clinics and a skilled nursing facility. In addition, a network of 11 independent community health centers (known as the San Francisco Community Clinic Consortium) relies on San Francisco General Hospital for specialty care services. The eReferral program was developed in response to long wait times for appointments at the hospital's specialty clinics.
Planning and Development ProcessKey steps included the following:
- Designing and vetting system: Collaboration with San Francisco General Hospital information technology (IT) staff allowed the program to design a secure and Health Insurance Portability and Accountability Act (HIPAA)–compliant electronic referral system. The proposed system was vetted with all medical directors of the primary care clinics.
- Implementing pilot project: The system was initially implemented in the hospital's gastroenterology clinic, where impact on patient wait times was monitored and tracked over the course of a year.
- Obtaining grant funding to expand program: Based on the success of the gastroenterology pilot, the San Francisco Health Plan (the local Medicaid managed care plan) awarded the hospital 3 grants totaling $1.5 million to spread the system to multiple medical and surgical specialty clinics and to magnetic resonance imaging, computed tomography, and ultrasound services.
- Securing buy-in from other specialty clinics: A core team of project leaders, including specialty champions, primary care champions, evaluation experts, and IT staff, identified early adopters and conducted informational presentations designed to convince additional specialty clinics to use the system. As part of this effort, the team created and distributed an orientation packet that outlined all requirements for clinics wishing to adopt the system.
- Training new adopters: As new specialty clinics came on board, project leaders trained those clinicians and staff who would be using the system. A week before implementation, the specialist reviewer(s) and schedulers received hands-on training in navigating the system and completing core tasks. During the first week of implementation, project leaders provided staff with additional training and technical support to ensure a smooth transition. This training process is repeated whenever a new clinic adopts the system.
- Conducting site visits to primary care clinics: Using additional grant money from the San Francisco Health Plan, project leaders conducted site visits with 18 primary care clinics that use the system. During these visits, the leaders reviewed key system functions with staff to ensure that all clinics used the system in a consistent, uniform manner, and elicited system-wide and site-specific barriers to use of the system.
- Providing option for user feedback: To make the program responsive to user suggestions, project leaders added a "Suggestion Box" to the system. Clinicians use this free text box to contact administrators about any technical or operational problems they may be experiencing and/or offer suggestions for improvement. Submissions are compiled, reviewed, and, if appropriate, used to develop entries for a list of "Frequently Asked Questions" and responses.
- Provider and reviewer training: While the system is generally intuitive and requires little formal training, the eReferral team is developing online videos to train both referring providers and specialist reviewers (updated September 2013).
Resources Used and Skills Needed
- Staffing: The core administrative team for the system includes a director (who spends 1 to 4 hours a week on the program), IT staff support (20 hours a week), a specialty lead, and a full-time program manager. All physician team members are University of California San Francisco faculty members based at San Francisco General Hospital and participate in the project as a part of their regular job responsibilities. Additional faculty members participated in the planning, implementation, and evaluation of the project, serving as IT consultants, evaluators, and liaisons to various departments and specialties. Participating primary care and specialty clinics use existing staff to operate the system on a daily basis, although the specialist reviewers designated by the specialty clinics receive compensation for fulfilling this role.
- Costs: Total program costs are difficult to estimate, because responsibility is spread widely across departments and specialties. Initial implementation costs can be significant, as evidenced by the size of the aforementioned grant funding needed to spread the program throughout the network of hospital-affiliated clinics. Ongoing operational and oversight costs include the salary and benefits for program staff, along with compensation to specialist reviewers at each participating clinic. A typical referral review takes between 5 and 15 minutes, with each participating clinic handling between 20 and 250 referrals per month.
Funding SourcesAgency for Healthcare Research and Quality; California Healthcare Foundation; San Francisco Health Plan; Kaiser Permanente Community Benefit
Getting Started with This Innovation
- Recruit primary care and specialty champions: Implementation of this type of system may meet with resistance from physicians and staff. Identifying trusted clinician leaders who enthusiastically support the program can help to obtain buy-in and encourage uptake.
- Brainstorm opportunities for improvement: During the upfront planning stage, conduct meetings and information-sharing sessions with key staff and leaders to explore the current institutional culture, identify opportunities to improve referral processes, and brainstorm strategies for realizing those improvements.
- Make system easy to use: Because many clinicians may not be comfortable using a new technology, the system should be as intuitive and easy to use as possible, and be supported by clear instructions and easily accessible technical support.
Sustaining This Innovation
- Set aside time for primary care providers to use system: Set aside dedicated time on a regular basis to allow primary care providers to complete referral requests and respond to ongoing communications with specialists.
- Introduce system to new hires: Describe and demonstrate the system to new employees as part of their orientation program. This step helps to make the system an integral part of the organization's workflow and culture.
- Secure sustainable funding to pay reviewers: Look beyond time-limited grants to secure funds that can be used to compensate specialist reviewers, being careful to set up a payment system that encourages better patient triage and provider education rather than making or deterring referrals.
Use By Other OrganizationsInformation provided in September 2013 indicates that the University of California San Francisco/San Francisco General Hospital team has received inquiries about the eReferral system from numerous organizations across the United States and from Canada, the United Kingdom, Europe and India. To date, Orange County, San Diego County, and Los Angeles County safety net systems have developed referral programs modeled after eReferral. Community Health Center, Inc., the largest network of community health centers in Connecticut, has implemented an eReferral system in partnership with the University of Connecticut. Outside the safety net, UCSF Medical Center, UCLA Health System and Brigham and Women's Hospital in Boston are implementing systems based on eReferral.
Kim Y, Chen AH, Keith E, et al. Not perfect, but better: primary care providers' experiences with electronic referrals in a safety net health system. J Gen Intern Med. 2009;24(5):614-9. [PubMed]
Felt-Lisk S, McHugh M, Howell E. Monitoring local safety-net providers: do they have adequate capacity? Health Aff (Millwood). 2002;21:277-83. [PubMed]
Felland LE, Felt-Lisk S, McHugh M. Health care access for low-income people: significant safety net gaps remain. Issue Brief Cent Stud Health Syst Change. 2004;84:1-4. [PubMed]
Knudtson ML, Beanlands R, Brophy JM, et al. Treating the right patient at the right time: access to specialist consultation and non-invasive testing. Can J Cardiol. 2006;22:819-24. [PubMed]
Levin A. Consequences of late referral on patient outcomes. Nephrol Dial Transplant. 2000;15(Suppl 3):8-13. [PubMed]
Shaw LJ, de Berker DA. Strengths and weaknesses of electronic referral: comparison of data content and clinical value of electronic and paper referrals in dermatology. Br J Gen Pract. 2007;57:223-4. [PubMed]
Kim-Hwang JE, Chen AH, Bell DS, et al. Evaluating electronic referrals for specialty care at a public hospital. J Gen Intern Med. 2010;25(10):1123-8. Epub 2010 May 29. [PubMed
Straus, SG, Chen AH, Yee H Jr, et al. Implementation of an electronic referral system for outpatient specialty care. AMIA Annu Symp Proc. 2011;2011:1337-46. Epub 2011 Oct 22. [PubMed
|Disclaimer: The inclusion of an innovation in the Innovations Exchange does not constitute or imply an endorsement by the U.S. Department of Health and Human Services, the Agency for Healthcare Research and Quality, or Westat of the innovation or of the submitter or developer of the innovation. Read more.|
Service Delivery Innovation Profile
Original publication: May 26, 2010.
Original publication indicates the date the profile was first posted to the Innovations Exchange.
Last updated: October 09, 2013.
Last updated indicates the date the most recent changes to the profile were posted to the Innovations Exchange.
Date verified by innovator: September 08, 2013.
Date verified by innovator indicates the most recent date the innovator provided feedback during the annual review process. The innovator is invited to review, update, and verify the profile annually.