Technical ReportPDF Available

Constrained Resource Education Scheduling (CORES): a new approach to complex scheduling



Scheduling, in the context of constrained resources, eg a limited number of sites or teachers, or other learner requirements, can become very complex. Standard computing algorithms deal poorly with such constraints. CORES takes a new approach that is scalable.
Constrained Resource Education Scheduling (CORES): a new approach to complex
Corey Wirun, System Architect, OHMES, University of Calgary
David Topps, Medical Director, OHMES, University of Calgary
This paper describes key factors in the design of an automated scheduling system within the
context of Community-Based Curriculum (CBC). CBC scheduling has unique characteristics
in the field of Medical Education because the concept of learner and learning events is
different than traditional classroom or course-centric scheduling: topic exposure is
determined by experiential factors; learners do not all follow the same trajectory; resources
are often much more geographically dispersed.(13) These differences make it more
technically challenging in that the dimensionality of the inputs to the schedule generation are
much more complex.
In addition, the mechanics of CBC schedule generation are equally reliant on workflows as
part of the scheduling process. These workflows can take the form of post hoc ‘what-if’
scheduling scenarios, where program secretaries and directors have to balance multiple
conflicting requests, and have just as much influence on the system technical design as the
mechanics of schedule generation itself.
At the most basic level, a medical education schedule (CBC, or otherwise) consists of a set
of learners and the distribution of those learners across learning events (rotations) in a
manner that meets defined schedule criteria. Schedule criteria are classified as hard or soft
- meaning that some criteria are critical to schedule acceptance (hard), whereas other
criteria are open to various levels of compromise (soft).
Rotations are often considered as hard constraints because they are dictated by academic
requirements. They may also include continuity-of-care characteristics such as contiguous
blocks, physical location restrictions, location re-usability, duration, or fixed/floating
placement within the academic year.
On the ‘softer’ side of scheduling criteria is the availability of the preceptors who preside over
rotations throughout the academic year.(4) Preceptors are responsible for declaring learner
capacity for a given learning event, i.e. how many learners can be supported for a rotation,
at a given learning location, and academic period?
Factoring in site or preceptor capacity in a medical education schedule is not trivial.
Capacity may vary over the academic year due to preceptor resilience or logistical
considerations. In addition, capacity may not be exact, but defined as a range for a learning
event, the desire being the incorporation of utilization targets that reflect learner throughput,
educational goals, or economic drivers.
Learner capacity falls into a ‘softer’ type of constraint because meeting capacity numbers
may not be critical to the schedule feasibility. For example, a capacity overrun for a specific
rotation at a specific location may not be a show-stopper but still has to be flagged as a
schedule exception. However, the schedule generation can still proceed with this exception
in place - for further examination by the scheduler.(5)
This differs from hard criteria violations, where the schedule generation should stop and
attempts to address the violation before continuing. If the schedule generation cannot
correct the violation, the entire schedule is thrown out.
At a certain point, the success, or fitness, of an automated schedule starts to become
questionable as the number of requirement dimensions increase. Also, the requirements will
inevitably start to overlap, where the complexity of an evaluation/prioritizing process rapidly
escalates. Of equal concern is that, as the requirements’ complexity increases, the ‘quality’,
or fit, of a generated solution decreases exponentially because the factors are too
constrained. Herein lies the major risk to scheduling system design: can enough of the
desired schedule constraints be accommodated to generate a schedule that is satisfactory
and practical for use?
Scheduling as an Automation Opportunity
The design of automated systems that support scheduling in the field of Medical Education
pose unique challenges.(69) To the software architect, it is a both a boon and a curse that
scheduling is a very active research topic - a simple Google search will uncover dozens of
academic papers that have been written on the subject. While these materials are helpful in
giving the system implementer an appreciation of the problem of scheduling, they will only
go so far.(10) There will still be significant design work to address the nuances of the
specific problem domain.
Effective scheduling algorithms, from a computer science perspective, have long been
considered to be of the non-deterministic variety. This class of algorithm states that for the
same set of inputs, consecutive iterations of a program on those inputs may yield completely
different results, or may not complete at all. These are classically hard problems for
computers to solve e.g. Traveling Salesman problem, the Knapsack Problem.
To many, this seems counter-intuitive as to why this approach is feasible software works,
or it does not. However, the paradigm of non-deterministic algorithms grew out of necessity
to address the limitations imposed by traditional deterministic algorithms.
As an illustration of a deterministic approach to schedule design, one might innocently
attempt a ‘divide-and-conquer’ methodology: for a given problem space, implement
individual schedule criteria in code with the end goal of meeting all schedule criteria at the
finish line. While, at the onset, it may be conceivable a solution will eventually be reached, it
is more likely that the complexity of the solution becomes prohibitive as constraints add up
and start compounding on each other. As a by-product of the increasing complexity,
performance starts to suffer as the solution has to do more and more work.
The rationale behind the initial gravitation to a deterministic approach could be attributed to
the fact that deterministic algorithms are often easier to visualize, and mistakenly, easier to
implement. Conversely, non-deterministic approaches tend to be more abstract (the
antithesis to the software developer mindset), and have a theoretical basis that requires a
deeper level of thinking to design and implement.
At the onset, it is vital that there be as complete an understanding as possible of the
complexity of the problem set. Unfortunately, diligent understanding of the problem still may
not guarantee success. This applies equally to deterministic and non-deterministic
approaches. It not uncommon in schedule automation development that, as functionality is
added (i.e. more constraints are addressed), the system abruptly stops improving in quality
or the performance of solution starts to decrease. This is especially true when exploring a
new scheduling domain where it is unknown at design time if success is achievable at all.
All that a software architect can do is develop the system in a manner where there is
flexibility in the design. The software must be flexible to accommodate, in the least invasive
manner possible, taking ‘steps backward’ in order to move forward again.
Schedule Workflows
Up to now, this paper has concentrated on schedule generation as a key activity in an
automated scheduling system. What appears to be under-appreciated in scheduling
literature is how schedules are managed after the schedule is generated and operational
(i.e. in-flight). This may be partly explained by the fact that these workflows are highly
domain specific and not of general interest. It may also be because these activities are more
deterministic, and thus fall below the radar of academic computing interest.
In any case, a system-generated schedule will undoubtedly be complex in nature. The
resulting relationships between rows and columns of the schedule will be extremely fragile
as a result of the application of its hard and soft scheduling criteria. Because of this, it is
highly doubtful that manual manipulation of the schedule will be able to maintain these
relationships. Automated tools, of at least the same complexity as the generation process,
will be required to manage the schedule once it is in-flight. It is impractical to expect manual
manipulation to be effective as it will likely be time intensive and will compromise schedule
In the context of CBC, scheduling systems must be agile enough such that schedule criteria
can be modified and re-applied at any point in the life of an in-flight schedule. Some
common in-flight changes could be:
· Addition and removal of learners (e.g. extended leave, off-cycle learners)
· Addition/removal/reallocation of learning locations (e.g. location unavailability due to
renovations, etc.)
· Changes in teaching capacity (e.g. preceptor availability changes)
Equally important is that in-flight changes must have a localizable effect minimizing the
invasiveness of the change on other locations, rotations, teachers or learners (the “innocent
bystanders”), to a scope deemed appropriate by the scheduler. For the purposes of this
paper, this capability is called Controllable Schedule Re-entrancy (CSR).
Controllable Schedule Re-entrancy
In the functional sense, CSR is defined as being able to take an existing schedule (or a
partial schedule), and having the ability to regenerate the schedule, in its entirety, or in-part,
using different schedule criteria. Unfortunately, schedule feasibility (i.e. how well the
schedule meets the criteria) now becomes a philosophically challenging topic. From an
algorithmic perspective, it may be complicated to infer an overall schedule feasibility when
different portions of the schedule are evaluated using different criteria. More striking,
however, are the potential ramifications of modifying the hard constraints and what the end
effect is on the schedule. Hard constraints, by definition, are mandatory, rigid declarations of
rules applicable to an entire schedule. One could argue that a schedule with different hard
constraints is a completely different schedule. To mitigate this problem, and still facilitate
CSR, limiting schedule criteria changes to soft constraints is a reasonable compromise.
The concept of CSR is one that is not readily found in scheduling academic literature. Most
literature concentrates on the definition and setup of schedule criteria in advance of
schedule generation. As examples, papers by Cohn et al,(6) and Topaloglu (9) describe
medical education scheduling problems very similar to CBC. In their papers, they refer to
the modelling of schedule criteria, in its entirety, before submission to the schedule
generation is performed. CSR is not practical in this scenario since schedule constraints
must be able to be dynamic and ad hoc - supporting an iterative scheduling process.
This leads to another important differentiator of this discussion from current literature. As
part of the CBC scheduling process, initial and in-flight schedule criteria are defined and
executed by the end-user, who may not be technically savvy. Presenting the scheduler with
an efficient, iterative scheduling process promotes the investigation of dynamic ‘what-if’
scheduling scenarios in a time-efficient fashion - without the need for technical/development
resources to reconfigure/model revised schedule criteria..
It makes sense that there is significant functional overlap between the processes involved in
initial schedule generation and those involved in schedule re-generation as a result of CSR.
In a pseudo-deterministic sense, schedule generation becomes a black box that takes an
initial schedule (empty, complete or partial), a set of criteria, and then generates a new
revision of the schedule for presentation to the user.. From a system design perspective,
schedule generation process becomes a reusable component, which has significant benefits
to system testability and sustainability.
For all intents and purposes, this solution becomes a blueprint for a generic scheduling
platform, versus a built-to-purpose program - and is much more scalable.
The project inspiring this paper was originally conceived as a schedule automation effort.
Significant business analysis time was spent taking inventory of hard and soft criteria which,
previously, existed only in spreadsheets.
As schedule constraints started to be implemented in code, solution technical
understandability started as being relatively simple, but became difficult as the schedule
criteria accumulated. Each new criteria required an increase in code complexity. There was
initial success using deterministic methods, but at a certain point, schedule viability suddenly
stalled and performance started to drop in attempts to accommodate. This breakdown was
attributed to a critical mass of constraints being reached that precluded the current design
methodology from being able to keep pace. In non-technical terms, the design painted itself
into a corner.
After a refactoring effort, it was found that non-deterministic algorithms ended up being much
more scalable. As the constraint set grew more complex, the platform proved itself to be
more agile to be able to accommodate the increased demands. This was attributed to the
observation that incremental increases in schedule criteria complexity yielded much smaller
increases in code complexity in non-deterministic systems versus deterministic ones.
The following table illustrates results taken at regular intervals as solution development
evolved. Initial solution development concentrated on deterministic algorithms for a limited
learner size. As criteria complexity and sample set size was increased, schedule
fitness/viability and system performance prematurely plateaued. Non-deterministic
approaches were then investigated and implemented.
Total Time
Fitness (%
Hardware acceleration added (multi-core computing)
Hardware upgrade to industrial RISC server
D - deterministic algorithm
ND - non-deterministic algorithm
Notes/conclusions from this table:
As the complexity of constraints increased, a plateau was reached using
deterministic algorithms where further complexity had a negative effect on
viability/fitness and no further progress was achievable.
Deterministic algorithms had linear performance related to number of learners,
whereas non-deterministic performance was not linear.
Physical hardware and hardware acceleration in software have a huge impact on
Non-deterministic algorithms showed improved performance and solution feasibility
It is interesting to note that, in the non-deterministic solutions, the variability of the schedule
fitness significantly varied between successive runs. This could easily be explained by the
prevalence of randomness in many non-deterministic algorithms.
Looking back at the methodology of the design process itself, the concept of CSR should
have been addressed much earlier in this phase. If it had, it might have disqualified the use
of the deterministic approach very early on - the prohibitive complexity designing both
aspects of schedule generation and CSR should have been immediately obvious.
When the methodology took on a non-deterministic track, having the dual goals of schedule
generation and schedule re-entrancy at the forefront lead to a much more informed decision
when it came time to selecting a non-deterministic approach.
Scheduling medical residents in Community-Based Learning is challenging both from a
technical as well as workflow perspective. The onus is on the solution implementer to
ensure that the technical design reflects not only the results of detailed schedule analysis,
but an understanding of the work processes involved after a schedule goes live.
Jeanine Robinson - Department of Family Medicine, Alberta Health Servicesfor subject
matter expertise and input on scheduling workflows.
1. Strasser RP, Lanphear JH, McCready WG, Topps MH, Hunt DD, Matte MC. Canada’s
new medical school: The Northern Ontario School of Medicine: social accountability
through distributed community engaged learning. Acad Med. United States;
2. Ford JC, Pinder KE, Ovalle WK, Li CH. Pathology education in a multisite urban/rural
distributed curriculum. Hum Pathol [Internet]. W.B. Saunders; 2008 Jun 1 [cited 2019
Mar 9];39(6):8116. Available from:
3. Flexner A. The Flexner Report: Medical Education in the United States and Canada
[Internet]. 1910 [cited 2019 Mar 9]. Available from:
4. Lewallen LP, DeBrew JK, Stump MR. Regulation and Accreditation Requirements for
Preceptor Use in Undergraduate Education. J Contin Educ Nurs [Internet]. SLACK
Incorporated; 2014 Aug 1 [cited 2019 Mar 10];45(9):38690. Available from:
5. Phan K, Brown S. Decreased Continuity in a Residency Clinic: A Consequence of
Open Access Scheduling. Fam Med [Internet]. 2009 [cited 2019 Mar 10];41(1):4650.
Available from:
6. Cohn A, Root S, Kymissis C, Esses J, Westmoreland N. Scheduling Medical
Residents at Boston University School of Medicine. Interfaces (Providence) [Internet].
2009 [cited 2019 Mar 6];39(3):18695. Available from: https://search-proquest-
7. Topaloglu S. A shift scheduling model for employees with different seniority levels and
an application in healthcare. Eur J Oper Res [Internet]. North-Holland; 2009 Nov 1
[cited 2019 Mar 6];198(3):94357. Available from: https://www-sciencedirect-
8. Smalley HK, Keskinocak P. Automated medical resident rotation and shift scheduling
to ensure quality resident education and patient care. Health Care Manag Sci
[Internet]. 2016 Mar 30 [cited 2019 Mar 6];19(1):6688. Available from: https://search-
9. Topaloglu S, Ozkarahan I. A constraint programming-based solution approach for
medical resident scheduling problems. Comput Oper Res [Internet]. 2011 Jan [cited
2019 Mar 6];38(1):24655. Available from:
10. Day TE, Napoli JT, Kuo PC. Scheduling the Resident 80-Hour Work Week: An
Operations Research Algorithm. Curr Surg [Internet]. Elsevier; 2006 Mar 1 [cited 2019
Mar 10];63(2):13641. Available from:
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
At academic teaching hospitals around the country, the majority of clinical care is provided by resident physicians. During their training, medical residents often rotate through various hospitals and/or medical services to maximize their education. Depending on the size of the training program, manually constructing such a rotation schedule can be cumbersome and time consuming. Further, rules governing allowable duty hours for residents have grown more restrictive in recent years (ACGME 2011), making day-to-day shift scheduling of residents more difficult (Connors et al., J Thorac Cardiovasc Surg 137:710-713, 2009; McCoy et al., May Clin Proc 86(3):192, 2011; Willis et al., J Surg Edu 66(4):216-221, 2009). These rules limit lengths of duty periods, allowable duty hours in a week, and rest periods, to name a few. In this paper, we present two integer programming models (IPs) with the goals of (1) creating feasible assignments of residents to rotations over a one-year period, and (2) constructing night and weekend call-shift schedules for the individual rotations. These models capture various duty-hour rules and constraints, provide the ability to test multiple what-if scenarios, and largely automate the process of schedule generation, solving these scheduling problems more effectively and efficiently compared to manual methods. Applying our models on data from a surgical residency program, we highlight the infeasibilities created by increased duty-hour restrictions placed on residents in conjunction with current scheduling paradigms.
Background: Nurse preceptors are widely used in pre-licensure RN education to facilitate the educational process. Often, these preceptors are staff nurses employed by clinical agencies. Currently, there are no standardized guidelines for qualifications, roles and responsibilities, or best practices. Method: Systematic website review was conducted of all Boards of Nursing and nursing accrediting bodies in the United States and Canada. Results: Seven categories of information were identified: preceptor qualifications, faculty and nursing program role, curriculum placement, written policies, ratios, orientation, and preceptor availability. Conclusion: Research is needed to document issues and current practices to create a model of best practices in preceptor orientation and use, as well as faculty supervision in prelicensure nursing education. J Contin Educ Nurs. 2014;45(x):xxx-xxx.
This paper addresses the problem of scheduling medical residents that arises in different clinical settings of a hospital. The residents are grouped according to different seniority levels that are specified by the number of years spent in residency training. It is required from the residents to participate in the delivery of patient care services directly by working weekday and weekend day shifts in addition to their regular daytime work. A monthly shift schedule is prepared to determine the shift duties of each resident considering shift coverage requirements, seniority-based workload rules, and resident work preferences. Due to the large number of constraints often conflicting, a multi-objective programming model has been proposed to automate the schedule generation process. The model is implemented on a real case in the pulmonary unit of a local hospital for a 6-month period using sequential and weighted methods. The results indicate that high quality solutions can be obtained within a few seconds compared to the manually prepared schedules expending considerable effort and time. It is also shown that the employed weighting procedure based on seniority levels performs much better compared to the preemptive method in terms of computational burden.
Persistent calls come from within the graduate medical education community and from external sources for regulating the resident duty hours in order to meet the obligations about the quality of resident education, the well-being of residents themselves, and the quality of patient care services. The report of the Accreditation Council for Graduate Medical Education (ACGME) proposes common program requirements for resident hours. In this paper, we first develop a mixed-integer programming model for scheduling residents’ duty hours considering the on-call night, day-off, rest period, and total work-hour ACGME regulations as well as the demand coverage requirements of the residency program. Subsequently, we propose a column generation model that consists of a master problem and an auxiliary problem. The master problem finds a configuration of individual schedules that minimizes the sum of deviations from the desired service levels for the day and night periods. The formulation of this problem is possible by representing the feasible schedules using column variables, whereas the auxiliary problem finds the whole set of feasible schedules using constraint programming. The proposed approach has been tested on a series of problems using real data obtained from a hospital. The results indicate that high-quality schedules can be obtained within a few seconds.
The chief residents in the psychiatry program at Boston University School of Medicine (BUSM) must construct a schedule that simultaneously assigns residents to five types of call shifts, spanning three different hospitals, over a 365-day planning horizon. We show how user expertise and heuristic approaches alone fail to find acceptable solutions to this complex combinatorial problem; likewise, mathematical programming techniques alone are inadequate, largely because they lack a clearly definable objective function. However, by combining both approaches, we were able to find high-quality solutions in a very short time. The resulting schedule, which BUSM uses currently, has yielded substantial benefits; the solution quality has improved, and the effort required to develop the solution has been reduced.
Like many rural regions around the world, Northern Ontario has a chronic shortage of doctors. Recognizing that medical graduates who have grown up in a rural area are more likely to practice in the rural setting, the Government of Ontario, Canada, decided in 2001 to establish a new medical school in the region with a social accountability mandate to contribute to improving the health of the people and communities of Northern Ontario. The Northern Ontario School of Medicine (NOSM) is a joint initiative of Laurentian University and Lakehead University, which are located 700 miles apart. This paper outlines the development and implementation of NOSM, Canada's first new medical school in more than 30 years. NOSM is a rural distributed community-based medical school which actively seeks to recruit students into its MD program who come from Northern Ontario or from similar northern, rural, remote, Aboriginal, Francophone backgrounds. The holistic, cohesive curriculum for the MD program relies heavily on electronic communications to support distributed community engaged learning. In the classroom and in clinical settings, students explore cases from the perspective of physicians in Northern Ontario. Clinical education takes place in a wide range of community and health service settings, so that the students experience the diversity of communities and cultures in Northern Ontario. NOSM graduates will be skilled physicians ready and able to undertake postgraduate training anywhere, but with a special affinity for and comfort with pursuing postgraduate training and clinical practice in Northern Ontario.
Open access scheduling decreases waiting time to see physicians by using same-day appointment scheduling. In primary care residency training, continuity of care may be difficult to preserve with this method of scheduling because requirements for rotations often results in residents being unavailable in their primary clinic practice. Our objective was to examine continuity of care in a family medicine residency clinic during a 1-year period prior to implementation of open-access scheduling and during a 1-year period after open access scheduling started. Two indices to measure continuity were used: the Usual Provider Continuity Index (UPC) and the Modified Modified Continuity Index (MMCI). The Mann-Whitney test was used to determine differences in the UPC and MMCI between groups. The mean UPC and MMCI scores decreased with open access scheduling. Mean UPC was 0.59 with traditional scheduling versus 0.55 with open access scheduling. Mean MMCI was 0.51 for traditional scheduling and 0.44 with open access. Continuity of care decreased in our clinic after implementation of open access scheduling. Our results have implications for all primary care residency training programs since one of the hallmarks of primary care is maintaining continuity in the physician-patient relationship.
The resident 80-hour work week requires that programs now schedule duty hours. Typically, scheduling is performed in an empirical "trial-and-error" fashion. However, this is a classic "scheduling" problem from the field of operations research (OR). It is similar to scheduling issues that airlines must face with pilots and planes routing through various airports at various times. The authors hypothesized that an OR approach using iterative computer algorithms could provide a rational scheduling solution. Institution-specific constraints of the residency problem were formulated. A total of 56 residents are rotating through 4 hospitals. Additional constraints were dictated by the Residency Review Committee (RRC) rules or the specific surgical service. For example, at Hospital 1, during the weekday hours between 6 am and 6 pm, there will be a PGY4 or PGY5 and a PGY2 or PGY3 on-duty to cover Service "A." A series of equations and logic statements was generated to satisfy all constraints and requirements. These were restated in the Optimization Programming Language used by the ILOG software suite for solving mixed integer programming problems. An integer programming solution was generated to this resource-constrained assignment problem. A total of 30,900 variables and 12,443 constraints were required. A total of man-hours of programming were used; computer run-time was 25.9 hours. A weekly schedule was generated for each resident that satisfied the RRC regulations while fulfilling all stated surgical service requirements. Each required between 64 and 80 weekly resident duty hours. The authors conclude that OR is a viable approach to schedule resident work hours. This technique is sufficiently robust to accommodate changes in resident numbers, service requirements, and service and hospital rotations.
To address concerns about regional physician shortages within British Columbia, the University of British Columbia began distributing its undergraduate medical curriculum across multiple campuses (ie, urban Vancouver, small urban Victoria, and rural Prince George) in 2005. The distribution of the pathology curriculum required meeting 3 specific challenges: (i) implementation of high-quality technologies to facilitate distribution; (ii) recruitment of pathologists to teach; and (iii) creation of an electronic pathology learning center. Technological needs were met by a state-of-the-art audiovisual system allowing simultaneous interactive didactic sessions across all 3 sites, and by the use of a digital "virtual slide" system. Recruitment of pathologist educators proved challenging owing to comparatively limited staffing levels at the rural site. A physical and virtual pathology learning center was developed to assist students in self-directed study. Student performance on pathology examinations has proven to be essentially identical pre- and post-distribution, and is equivalent across all 3 sites. Quantitative and qualitative student survey data show that distributed pathology instruction is overwhelmingly well received by medical students at all sites, that pathologists' expertise is very important to students, and that pathology is one of the most popular components of the distributed curriculum. Pathology education continues to be a vital part of a distributed undergraduate medical program, and student grades and feedback demonstrate the value of the teaching and the technologies we have used. To be implemented successfully, the distribution of pathology education requires considerable financial and infrastructure investment, and ongoing commitment from pathologists and university administrators.