Technical Discussion

This page contains the history of items debated between developers. It could stand to be cleaned up a little bit. Perhaps we need to break some of the larger discussions into separate pages?

Action Items

There are no actions items at this time.

Discussion Items

Auditing / Event Broadcasting

Justin and I were talking about the intersection between auditing / event broadcasting and transactions. Here are some thoughts: Considering these functions

PatientService.createPatient(Patient p) {
     dao.saveOrUpdate(p);
     context.postMessage(new PatientCreatedEvent(p));
 }
 ObsService.createObs(Obs o) {
     dao.saveOrUpdate(o);
     context.postMessage(new ObsCreatedEvent(o));
 }

Assume we want to do a batch creation of patient + several obs like this:

// START TRANSACTION
 context.getPatientService().createPatient(p);
 for (Obs o : obsBatch) {
     context.getObsService().createObs(o);
 }
 // END TRANSACTION

If we truly want to guarantee that the application messages we're delivering are correct, then we need to meet two criteria:

1. Don't post messages until all database writes in the transaction are committed. (I.e. if a call to createObs() fails, then don't post the PatientCreatedEvent message.)
2. Failure to deliver a message should cause the whole transaction to rollback. (I.e. if we can't deliver the PatientCreatedEvent message, then don't create the patient.)

I think we can meet the first criterion under our current framework, although it informs how we'd handle transactions. (I'm not 100% sure this works, but I think that when calling context.postMessage() we can check whether there's an active transaction, and if so hold, queue up that message. Committing the transaction would have to cause a callback to the context, so it knows to actually post the queued messages.)

The second criterion is much more complicated. Justin points out that to meet it we'd need to be running an application server that does Container Managed Transactions like JBoss. (This strikes fear into my heart.)

So I suppose there are 3 options here:

1. Event notification is unreliable and you should expect occasional phantom messages.
2. Event notification is mostly reliable, but if the messaging system itself fails, then actions can happen without notifying listeners.
3. Event notification is 100% reliable: every action guarantees its listeners are notified.

I like #2 myself. Any thoughts?

Djazayeri 09:23, 16 December 2005 (US Eastern Standard Time)

Message Service API

The Message Service API will comprise the main message transport service. It should be used as the underlying service for all messaging in the system. For example, the Note Service API will use the Message Service API when message are send via email, phone, pager.

Methods

void sendMessage(Message message) throws SendMessageException;

Implementing Class (example)

public class EmailMessageServiceImpl implements MessageService {  
 
    public void sendMessage(Message notification) { 
      try { 
        Context context = new InitialContext();
        Session mailSession = (Session) context.lookup("java:/Mail");
        Message mailMessage = new MimeMessage(mailSession);		
        for ( User user : notification.getRecipients() )  {
          mailMessage.setRecipients(javax.mail.Message.RecipientType.TO, InternetAddress.parse(user.getEmail(), false) );
          mailMessage.setFrom(InternetAddress.parse("info@openmrs.org", false));
          mailMessage.setSubject(notification.getSubject());
          mailMessage.setText(notification.getMessage());
          Transport.send(mailMessage); // send message (probably want a try-catch around this ... left out for brevity)
        }
      } catch (Exception e) { 
        e.printStackTrace();
        throw new SendMailMessageException(); // subclass of SendMessageException
      }
    }
  }

Alert API

Over the past few days we have had several conversations about messaging and alerting. We have been cognizant of the fact that there are some subtle differences, but have tried to keep the same model for both types of messages. However, after a little more discussion today, we've decided (at least for the time being) to follow your advice and discern the two.

Our conversations today lead us to the position that Alerts are very closely related to Notes and User-Reported Errors. We checked the data model and confirmed that Alerts can be implemented as Notes. The only difference between Notes/Errors and Alerts are:

• A Note is entered manually (by a data entry clerk) to report an error or concern with a patient, observation, and/or encounter, while an Alert will tend to be automatically generated (in the case of the decision support system).
• An alert has a list of people who need to be actively informed of it, while a note should just be seen by anyone viewing a patient/obs page. (Anyone viewing a patient/obs should see related alerts too.) An error is sort of like an alert-to-the-data-manager.
• An alert is actionable, while a note isn't. (An error is probably actionable as "fixed")

So we're planning to model the alerting system around the Note.

Methods

List<Alert> getAlerts(User user);                       // Get alerts by user
  Alert getAlert(Integer alertId);                        // Get alert details  
  AlertCount getAlertCount(User user);                    // Get the alert count (contains information like #  read vs # actioned vs # all alerts)
  List<Alert> getAlertsByPatient(Integer patientId);      // Get alerts by patient
  List<Alert> getAlertsByObservation(Integer obsId);      // Get alerts by observation 
  List<Alert> getAlertsByEncounter(Integer encounterId);  // Get alerts by encounter
  void raiseAlert(Alert alert);                           // Raise an alert

Model

NoteRecipient

 user_id
 group_id
 relationship_type
 relationship_target
 view_date
 action_status ('none', 'actioned', ...) maybe defined in template
 action_date

An alert is (a Note) + (Possible Actions, Current Action Status) + (People who need to know about it)

 Alert 
   Note - associated with a patient, observation, and/or encounter
   AlertType
   ActionTaken (move to Note?)

 AlertType
   Name
   Description
   Template

 Template - used to generate the body for the alert
   Name
   TemplateText

Action-related

 Action - actions that can be taken for an alert
   Name
   Description

 PossibleAction - associated possible actions that can be taken for an alert
   Alert
   Action

Notification-related: we want to be able to send an alert to

• a specific user
• a specific group
• a specific role
• everybody with a certain relationship to a certain patient (e.g. MD-in-charge of patient_id 1234)

 NoteRecipient
   // each row should have either UserId OR GroupId OR RoleId OR (RelationshipType AND RelationshipTarget)
   NoteRecipientId (PK)
   NoteId (FK to Notes table)
   UserId
   GoupId
   RoleId
   RelationshipType
   RelationshipTarget

 AddressBookEntry
   // named distribution lists of NoteRecipients, maybe with a default value in the alert-type table for convenience, but not directly referenced by specific alerts

--Jmiranda 17:01, 23 January 2006 (US Eastern Standard Time)

Transactions

Considering the issue of doing transactions programmatically, consider this contrived example:

PatientService.createPatient(Patient p) {
    dao.startTransaction();
    dao.saveOrUpdate(p);
    dao.endTransaction();
 }
 ApplicationCode.someMethod(List<Patient> patientBatch) {
    dao.startTransaction();
    for (Patient p : patientBatch) {
        context.getPatientService().createPatient(p);
    }
    dao.endTransaction();
 }

These need to be nested transactions in the sense that the endTransaction() call in PatientService.createPatient() should not end the "higher-level" transaction initiated by ApplicationCode.someMethod().

Modifying PatientService.createPatient() as follows doesn't work:

PatientService.createPatient(Patient p) {
    dao.startOrContinueTransaction();
    dao.saveOrUpdate(p);
    dao.maybeEndTransaction(); // what does this mean?
 }

My book gives this as Spring's way of doing this programmatically:

TransactionDefinition td = new DefaultTransactionDefinition(); // this means PROPAGATION_REQUIRED
 TransactionStatus tranStatus = transactionManager.getTransaction(td);
 try {
     //something
 } catch (ApplicationException e) {
     transactionManager.rollback(tranStatus);
     throw e;
 }
 transactionManager.commit(tranStatus);

The proper solution seems to be using Java 1.5 annotations to do transactions declaratively, and letting Spring work its magic. So something like:

@TxAttribute(propagationType=PropagationType.REQUIRED)
 PatientService.createPatient(Patient p) {
    dao.saveOrUpdate(p);
 }
 @TxAttribute(propagationType=PropagationType.REQUIRED)
 ApplicationCode.someMethod(List<Patient> patientBatch) {
    for (Patient p : patientBatch) {
        context.getPatientService().createPatient(p);
    }
 }

Also on the topic I'm going to transcribe a couple of insightful paragraphs from my Spring book: "We recommend applying transactions at the business layer. This allows the business layer to catch any exceptions that caused a rollback and issue the appropriate business-level application exceptions. ... Data access code should not delimit transaction boundaries because it doesn't implement business logic, and atomicity is a business-level concept. Let's say we apply a transaction on a data access method that updates the balance of an account. This would prevent us from reusing this method in a task that transferred data between two accounts. If the subtraction from the first account was already committed, then we could not roll back all changes in the case of the addition to the second account failing."

Djazayeri 16:07, 15 December 2005 (US Eastern Standard Time)

Selecting Obs

Hi Vibha,

To follow up from the call today (and mostly repeat what I emailed before) we will need something like this method:

 List<Obs> findObs(Patient who, Concept what, Clause c, Date fromDate, Date toDate, GroupMethod m);

"Clause c" needs to be able to represent the following:

1. For numeric, text, and datetime values
   • { <, <=, >, >=, =, != } a constant number/string/date
   • { between, not-between } two constant numbers/strings/dates
   • { in, not-in } a collection of numbers/strings/dates
2. For numeric values
   • above_hi_critical, above_hi_normal, below_low_normal, below_low_critical, outside_normal_range, outside_critical_range, inside_normal_range, inside_critical_range
3. For coded (i.e. concept) values
   • { =, != } a concept
   • { in, not-in } a collection of concepts
4. For boolean values
   • { =, != } a constant boolean
5. True (i.e. just get all obs of that type, regardless of value)

"GroupMethod m" needs to be able to represent: ALL_MATCHES, EARLIEST_MATCH, LATEST_MATCH

We’ll also want the same method that takes a ConceptSet as its second argument. And we’ll really want to have a version that takes a Collection<Patient>. I know you don’t need that.

So…any thoughts?

-Darius

Darius,

Here's a thought for grouping clauses...you could use "and()" and "or()" methods in the Clause class:

class Clause {
    public static final Clause TRUE = new Clause("=", new Object[] {Boolean.TRUE});
    public static final Clause NOT_EQUAL = "!=";
    // etc.
    // for convenience for operators with one argument
    public Clause(String operator, Object arg) { ... }
    // for operators that have multiple arguments (e.g., BETWEEN)
    public Clause(String operator, Object[] args) { ... }
    // allow for grouping of clauses
    public Clause or(Clause c) { ... }
    public Clause and(Close c) { ... }
  }

to allow for:

Clause A = new Clause(Clause.LESS_THAN, Double.valueOf(3.5));
  Clause B = new Clause(Clause.BEFORE, DateFormat.parse("1/1/2005"));
  Clause C = new Clause(Clause.NOT_EQUAL, Double.valueOf(1.5));
  (A and B) or C becomes (A.and(B)).or(C)
  A and (B or C) becomes A.and(B.or(C))

How would you handle a query like this?

 get the highest creatinine before first CD4 < 200

in two steps?

Okay, I'm going to step way outside of the box for a second.

What if the findObs() method used a generic "ObsQuery" object instead, e.g.:

 Obs<List> findObs(Patient who, ObsQuery query);

ObsQuery could be called recursively to build sophisticated queries. The resulting ObsQuery could be passed to the DAO level to generate the appropriate SQL.

class ObsQuery {
    public static ObsQuery query(ObsQuery q) { ... }
    public static ObsQuery query(Concept c) { ... }
    public static ObsQuery query(List<Concept> c) { ... }
    ...
    public ObsQuery before(Date d) { ... }
    public ObsQuery before(ObsQuery q) { ... }
    ...
    public ObsQuery between(Date start, Date finish);
    ...
    public ObsQuery lt(double n);  // op="<", operands=self & n
    public ObsQuery lte(double n); // op="<=", operands=self & n
    ...
    public ObsQuery first(); // op=first, operands=self
    public ObsQuery last();  // op=last, operands=self
    ...
    public ObsQuery and(ObsQuery q); // op=and, operands=self & q
    public ObsQuery or(ObsQuery q);  // op=or, operands=self & q
    ...
    // you could event support a generic operator
    public ObsQuery op(String operator, Object[] operands);
    // and support Clause
    public ObsQuery clause(Clause c); // op=c.op, operands=c.args
    ...
    // for DAO level to traverse the query
    public String getOperator();   // action for this part of query
    public Object[] getOperands(); // could be ObsQuery, Concept, Date, etc.
  }

Some examples:

"get most recent concept"

findObs(who, new ObsQuery(concept).last());

"get concepts between fromDate and toDate"

findObs(who, new ObsQuery(concept).after(fromDate).before(toDate));

"get max creatinine before CD4 < 200"

findObs(who, new ObsQuery(creatinine).before(new ObsQuery(CD4).lt(200).first()).max());

So, your proposed method could be implemented as:

List<Obs> findObs(Patient who, Concept what, Clause c, Date fromDate, Date toDate, GroupMethod m) {
    return findObs(who,
       new ObsQuery(what).clause(c).after(fromDate).before(toDate).group(m));
  }

-Burke 18:33, 16 December 2005 (US Eastern Standard Time)

Voided Bit

I've noticed that tables in the data model have the "voided" bit, but I'm
curious as to why it's nullable.  Shouldn't a row need to be voided or not
voided?  And doesn't that make for extra work when selecting because the
where clause has to say "where voided is not null and voided=0", when really
it should just have to say "where voided=0"?
-Christian

Agreed.
-Ben 09:33, 12 December 2005 (US Eastern Standard Time)

Concepts within the PATIENT table

This is in OpenmrsConstants.java. I'm vehemently opposed:

public static final Map<String, String> CIVIL_STATUS() {
	HashMap<String, String> civilStatus = new HashMap<String, String>();
	civilStatus.put("1", "Single");
	civilStatus.put("2", "Married");
	civilStatus.put("3", "Divorced");
	civilStatus.put("4", "Widowed");
 
	return civilStatus;
}

Generally speaking the problem is that we have something that should really be a concept, being stored in the Patient table. A couple possible solutions:

• Get civil_status out of the Patient table, and make it an Obs.
• Implement this as a Java 1.5 Enum.
• Put the meaning in the message.properties file

Djazayeri 17:14, 13 February 2006 (US Eastern Standard Time)

Noting errors

I just added the "if (role == null) { ... }" to the following code in Context.hasPrivilege(String). I figure this will be a one-time error for each developer who's using an older test database. Does anyone care how we mark these errors?

• Should I email the fix to the developers list?
• Should (someone) have emailed the list saying "update to the latest demo database"?
• Should I put a bad-style comment like I just did?
• Should I put in a log.warn instead?

Role role = getUserService().getRole(OpenmrsConstants.ANONYMOUS_ROLE);
 if (role == null) {
 	throw new RuntimeException("Database out of sync with code: " + OpenmrsConstants.ANONYMOUS_ROLE + " role does not exist");
 }
 if (role.hasPrivilege(privilege))
 	return true;

Djazayeri 17:30, 13 February 2006 (US Eastern Standard Time)

I appreciate the note. I've followed similar syntax with another change I made. In the future, we'll try and either have sql updates available and/or a note sent to the developer's list. Ben 16:04, 27 February 2006 (US Eastern Standard Time)

Export Patient XML Format

<patient_data>
     <patient
          patient_id="123"
          given_name="Darius" middle_name="Graham" last_name="Jazayeri" last_name2="Xyz"
          gender="M" race="Xyz" birthdate="1978-04-11 00:00:00" birthdate_estimated="false"
          birthplace="Atlanta" citizenship="US" tribe="tribeName"
          mothers_name="Mary" civil_status="Divorced" death_date="..." cause_of_death="..."
          health_district="Massachusetts" health_center="Cambridge Hospital" health_center_id="321">
         <names>
             --One name is picked randomly to go in the attributes above. All names including that one will go here also
             <name given_name="Darius" middle_name="Graham" last_name="Jazayeri" last_name2="Xyz"/>
         </names>
         <encounters>
             <metadata>
                 <location location_id="1">BIDMC Primary Care</location>
                 <encounter_type encounter_type_id="1">Physical Exam</encounter_type>
                 <form form_id="1">Physical exam form</form>
                 <provider provider_id="1">Bob, MD</provider>
             </metadata>
             <observations>
                 <obs obs_id="1" concept_id="1" concept_name="Concept name in session locale" datetime="2006-02-13 15:45:00"
                    accession_number="abc123" comment="a comment" date_started="..." date_stopped="..."
                    obs_group_id="1" value_group_id="1"
                    value_coded_id="1" value_coded="name of concept" datatype="coded" --only one of these will actually exist
                    value_boolean="true" datatype="boolean"                           --only one of these will actually exist
                    value_datetime="2006-02-14 14:23:05" datatype="datetime"          --only one of these will actually exist
                    value_numeric="5.3" datatype="numeric"                            --only one of these will actually exist
                    value_text="some text" datatype="text"                            --only one of these will actually exist
                    value_modifier="...">
                       The non-null one of the above values goes here, preceded by value_modifier, if any
                 </obs>
             </observations>
             <orders>
                 <order concept_id="1" instructions="as needed" start_date="2006-02-14 02:55:00" auto_expire_date="..."
                    orderer="1^Bob, MD" discontinued="true" discontinued_date="..." discontinued_reason="...">
                         Concept name of this order
                 </order>
             </orders>
         </encounters>
         <observations>  --these are observations that don't belong to any encounter
             same format as observations in encounters
         </observations>
     </patient> 
 </patient_data>

Djazayeri 03:07, 14 February 2006 (US Eastern Standard Time)

Drug data model

I was looking at the drug data table, and the following thoughts spring to mind:

• Are these one-to-one with concepts, or would "AZT 300mg tablet" and "AZT 100mg tablet" both share the concept "AZT"?
  • If so:
    • How are we dealing with translations of this?
  • Otherwise:
    • Why do we need drug_id instead of just concept_id?
    • Why do we have a name column

The answer is "yes" — that means we want to support both ways. In smaller settings where docs are ordering straight from the inventory, you'd have a concept for every drug form (and drug_id and name are less useful). In larger scale settings, however, we'd like to have AZT as the concept and each dosage form in the drug table. Basically, the drug table represents the "pharmacy inventory" and whether or not these match one-to-one with concepts is an implementation decision. We can discuss how we imagine handling this. -Burke

• Should daily_mg_per_kg really be in this table? If so, some prescribing guidlines say something like "20-25 mg/kg", and we probably want to represent that too.

Agreed. We're not excited about how this is currently modeled. I think the daily_mg_per_kg was a carry-over from PIH's MDR TB stuff. We need to have dosage, frequency, and concentration information...we can discuss options during the conference call. -Burke

• Why is combination a smallint(5)?

Should be tinyint (boolean equiv). This table has not been groomed as much as others, since we haven't started using it other than for a few brand drugs for AMRS. -Burke

• What's the distinction between inn and name?

inn was carried over from discussion with Hamish about the PIH model. inn is supposed to represent the international standard name for the active ingredient(s). For AMRS, we plan on using concepts for this (i.e., linking drug forms through the same concept_id); however, I suppose if you decided to create a concept-per-drug-form one-to-one mapping between drugs and concepts, then you may still want a mechanism to relate all of your forms of AMPICILLIN. Personally, I'd rather folks do that with concept_set and ditch the inn attribute. We can discuss options. -Burke

• We have boolean columns for "important?" and "voided?"
  • "important?" is probably webapp functionality than doesn't belong in the core data model, but one should be able to void drug formulations.

I don't know what "important" means, but we'd agree that changed_by, date_chagned, voided, voided_by, date_voided, and voided_reason should be added to the drug table. -Burke

Djazayeri 15:59, 27 February 2006 (US Eastern Standard Time)

Application properties

I want to define some application-level properties that are going to differ between the PIH and Regenstrief systems. For example the PIH patient-set-selector tool will allow the user to select patients based on a concept called "ARV TREATMENT GROUP" which Regenstrief won't have. Any strong feelings about where this type of configuration should go?

• I assume we should have separate config properties files for each module.
• Should we structure each module so it has a base properties file, and then a second properties file that we/you use to override those shared values?

Djazayeri 17:59, 13 March 2006 (Eastern Standard Time)

Paul and I were talking about this recently. I agree that we need to both support implementation-specific settings and module-specific settings. Additionally, I agree that each should have a "default" set of properties and a defined method for extending/adjusting those properties. Site specific changes (other than simply overriding properties iin OpenMRS-build.properties as we do now) could be treated almost like just another module — i.e., the same process we define to customize & separate modules from the base could be extended to implementation-specific changes by simply choosing a unique "module" name for each site (e.g., amrs or hiv-emr). The compile process needs a way to detect, at compile time, which modules are desired in the WAR and dynamically include them. I've got more thoughts on this, but don't have time. I'll add this topic for this Thursday's conference call. -Burke 18:20, 13 March 2006 (Eastern Standard Time)

Task Scheduling

Here's a very crude attempt at coding what I've been thinking. I'm having a little bit of difficulty picturing how we plug the different frameworks (jdk timer, quartz) into our application and will need to spend some more time tomorrow trying to finish the puzzle. The more concrete elements are at the bottom of the page where i try to show how we might create a ProcessFormEntryQueueTask to fulfill our current needs.

I have been struggling with the "Why don't we just adopt one of these frameworks" idea, but I really want this to be an elegant solution that allows us some flexibility, so I'm not giving up yet.

As you can see, the scheduler service is an interface. Implementations might include one for the JDK timer (SimpleSchedulerService), one for Quartz (QuartzSchedulerService), and maybe one for a Quartz persisted service (PersistentQuartzSchedulerService). I haven't thought this through completely, so bear with me.

Scheduler Service

package org.openmrs.scheduler;
 
  /**
   *  Defines methods required to schedule a task.  
   */
  public interface SchedulerService { 
 
    /**
     *  Schedule a recurring task that occurs according to the given schedule (start time and interval).
     */
    public void scheduleTask( Task task, Schedule schedule );
 
  }

Task

package org.openmrs.scheduler;
 
  /**
   *  Stateless task interface 
   */
  public interface Task { 
 
    /**
     *
     */
    public void execute( );
 
  }

Stateful Task

package org.openmrs.scheduler;
  /**
   *  Stateful task interface
   */  
  public interface StatefulTask extends Task { 
 
    /**
     *  Execute a task with the given state (in case the process will need some 
     *  data from the database (via the services layer).
     */
    public void setContext(Context context);
 
    // Not sure if we need it, but might be a good idea.
    //public void setData(Map data);
  }

Schedule

package org.openmrs.scheduler;
 
  /**
   *  Describes when to start a task and how often it is executed.
   */
  public class Schedule { 
 
    private long start;
    private long interval;
 
    /** 
     *  Public constructor
     *  
     *  @param start	timestamp for when to start the task (does not need to be in the future if the interval is specified).
     *  @param interval   time to wait between executing task (<= 0 indicates that it should only be run once)
     */
    public Schedule(long start, long interval) { 
      this.start = start;
      this.interval = interval;
    }
 
 
    /**
     *  Get the start time (in seconds) when the task should be executed.  
     *  For instance, use "new Date().getTime()", if you want it to start now.
     *  
     *  @return long  start time
     */
    public long getStart() { 
      return start;
    } 
 
    /**
     *  Get number of seconds until task is executed again
     *  
     *  @return long  number of seconds.
     */
    public long getInterval() { 
      return interval;
    }
  }

Simple Scheduler Service

package org.openmrs.scheduler;
 
  import java.util.TimerTask;
  import java.util.Timer;
 
  /**
   *  Simple scheduler service that uses JDK timer to trigger and execute scheduled tasks
   */
  public class SimpleSchedulerService { 
 
    /**
     *  Schedule the given task according to the given schedule.
     */
    public void scheduleTask( SimpleTimerTask task, Schedule schedule ) { 
      Timer timer = new Timer();
      if ( schedule.getInterval() == 0 ) { 
        timer.schedule(task, schedule.getStart() );
      } else { 
        timer.schedule(task, schedule.getStart(), schedule.getInterval() );
      }
    }
 
 
  }

Simple Timer Task

package org.openmrs.scheduler;
 
  import java.util.TimerTask;
 
  public class SimpleTimerTask implements TimerTask { 
    // 
    private Task task;  
    public SimpleTimerTask(Task task) { 
      this.task = task;
    }
 
    public void run() { 
      task.execute();
    }
  }

Say Hello Task

/**
   *  Implementation of the stateless task.
   */
  public class SayHelloTask implements Task { 
 
    /**
     *  Illustrates stateless functionality as simply as possible.  Not very 
     *  useful in our system, except maybe as a polling thread that checks internet
     *  connectivity by opening a connection to an external URL. 
     *
     *  But even that isn't very useful unless it tells someone or something 
     *  about the connectivity (i.e. calls another service method)    
     */
    public void execute() { 
      System.out.println("Hello World!");
    }
  }

Process Form Entry Queue Task #1

/**
   *  Implementation of the stateful task that simply gets the next form entry out of the queue
   *
   */
  public class ProcessFormEntryQueueTask implements StatefulTask { 
 
    // Instance of context used during task execution
    private Context context;
 
    /**
     *  Setter method for context
     */
    public void setContext( Context context ) { 
      this.context = context;
    }
 
    /** 
     *  Process the next form entry in the database and then remove the form entry from the database.
     */
    public void execute() {
      FormEntryQueue queue = context.getFormEntryService().getNextFormEntryQueue();
      FormEntryQueueProcessor processor = new FormEntryQueueProcessor( context );
      processor.transformFormEntryQueue( queue );
 
      // This actually happens in transformFormEntryQueue ... wondering if client should worry about this instead.
      //context.getFormEntryService().deleteFormEntryQueue( queue );
 
    }
 
  }

Process Form Entry Queue Task #2

/**
   *  Implementation of a stateless task that gets the next form entry and processes it.
   */
  public class ProcessFormEntryQueueTask implements StatefulTask { 
 
    // Instance of context used during task execution
    private Context context;
 
    /**
     *  Setter method for context
     */
    public void setContext( Context context ) { 
      this.context = context;
    }
 
    public void execute() { 
 
      // Or instead of doing all that, you could look at the FormEntryQueueProcessor 
      // class and realize that the transform next entry has already been implemented.  :)
      new FormEntryQueueProcessor( context ).transformNextFormEntryQueue();
 
    }
  }

--Jmiranda 02:36, 16 March 2006 (Eastern Standard Time)

Justin, this is a great start. Reponding to your comment above about deleting queue entries in the processor vs. the calling method...in its current state, calling processFormEntryQueue(FormEntryQeueu feq), it might make sense to only do the processing and leave it to the caller to delete the entry. In fact, I'd rather put that code into a FormEntryQueueProcessor.processNextFormEntryQueue() method, so the schedule would just call that method and the business of deleting queues after they're processed is left to the processor.

This brings up another point: ideally, we create the FormEntryQueueProcessor object only once per context. I guess thats could be up to FromEntryQueueProcessor if I added a .getInstance(Context myContext) method.

I googled an older arcticle on the topic of task scheduling and another slightly more recent one. You may have seen them & I don't know if they contain anything particularly insightful.

I think we could survive being bound to quartz, but if you can create an abstraction layer, that'd be great!

-Burke 12:15, 16 March 2006 (Eastern Standard Time)

Sorry, I added this section yesterday, but it was up with the rest of my rambling above, so you may have missed it. I just cut-and-paste it down here to give the conversation a better flow.

The more I think about it, the more I realize that synchronization should be handled by the code that is being executed, not by the scheduler or task components. The reason is because synchronization is a requirement of the use case that we were discussing (i.e. we don't want to process the same form twice). This is a business rule. We might have a task that we want to schedule twice at different times/intervals and we may not care whether they are synchronized (unfortunately, I cannot think of an example). If we synchronize at a level higher than the code that needs to be synchronized (i.e. getting next entry queue within FormEntryQueueProcessor.processFormEntryQueue() method) we might significantly degrade performance. In addition, I think we risk running into deadlock situations (although I can't envision a scenario at the moment) the higher up we synchronize. I think we need to block on the call to getNextFormEntryQueue(), right? Or maybe have a synchronized processNextFormEntryQueue() method that looks like this ...

public synchronized void processNextFormEntryQueue() { 
    FormEntryQueue formEntry = context.getFormService().getNextFormEntryQueue();
    transformFormEntryQueue( formEntry ); 
    context.getFormService().deleteFormEntryQueue( formEntry )
  }

--Jmiranda 08:54, 17 March 2006 (Eastern Standard Time)

Here's another issue I had written about yesterday ... and then tried to clarify (read: obfuscate) this morning.

One of the issues with scheduling that we may need to be careful about is that fact that we only want one instance of a task to be run for a system (i.e. email report daily should only run once per day for the entire system). If we utilize clustering or design some complex server hierarchy, we need to make sure that only one of the servers instantiate any given task. We may even need to think about designating a single server as the scheduling server, and make sure all scheduling goes through that server. That seems a bit overly complex to me right now, but I was thinking about the bigger picture and I got a little freaked out about issues we may run into having multiple systems sharing information.

The complex server hierarchy doesn't scare me as much as the clustering issue. I envision the following scenario. We have 4 clinics in a project. Those 4 clinics needs to send daily email reports with statistics compiled using their local database. We have a fifth node that handles all the data and needs to send a weekly report on the entire project. In this case, we simply schedule the daily emails on the 4 child nodes, and a weekly report on the parent node. I don't see any issues with that, related specifically to task scheduling. The data synchronization issue is problematic, but that's an issue for the Data Synchronization and Complex Server Hierarchy teams. Oh wait, that's us. But at least that's far enough into the future for us not to worry about at the moment. Just thought we should have some of these issues in the back of our minds.

Clustering will be an issue though. If we decide we want to use clustering at some point, we need to make task scheduling "cluster-aware". I have no idea how Tomcat handles clustering and don't really think it's worthwhile to pursue that research at the moment. But if you guys have any plans to go down that route, let's have another discussion.

--Jmiranda 09:18, 17 March 2006 (Eastern Standard Time)

User and Patient extend Person

I know you guys have given a lot of thought to this already and will probably have much more to say, but I wanted to throw down some use cases we have that would support this model and also help us think about implementation challenges. Most of the use cases stem from the fact that we have a provider called an "accompagnateur" (which must be said in a snobby french accent) that we often want to track almost as rigorously as a patient:

• We need to be able to locate accompagnateurs by address
• We need to be able to issue/search on Identifiers for accompagnateurs
• At some point in the not-so-distant future, we will need to schedule both patients and accompagnateurs to come on certain days and track whether or not they showed up
• We very well may want to eventually be able to make observations about accompagnateurs or other health care providers.
• This may not affect the model at all, but we will want to be able to put together a drug order based on an accompagnateur, who may have 5 patients that they are going to visit and bring medication to.

Again, these are just examples to think about.

Also, I decided to do a quick check of how MySQL felt about FKs referencing FKs, and it seems to be ok with them. Here are my tests:

CREATE TABLE `test_person` (
  `p_id` int(11) NOT NULL DEFAULT '0',
  `name` varchar(255) NOT NULL DEFAULT '',
  PRIMARY KEY  (`p_id`)
) engine=innodb DEFAULT charset=utf8;

CREATE TABLE `test_patient` (
  `p_id` int(11) NOT NULL DEFAULT '0',
  `location` varchar(255) NOT NULL DEFAULT '',
  PRIMARY KEY  (`p_id`),
  constraint `patient_person_id` FOREIGN KEY (`p_id`) REFERENCES `test_person` (`p_id`)
) engine=innodb DEFAULT charset=utf8;

CREATE TABLE `test_user` (
  `p_id` int(11) NOT NULL DEFAULT '0',
  `username` varchar(255) NOT NULL DEFAULT '',
  PRIMARY KEY  (`p_id`),
  constraint `user_person_id` FOREIGN KEY (`p_id`) REFERENCES `test_person` (`p_id`)
) engine=innodb DEFAULT charset=utf8;

CREATE TABLE `test_role` (
  `role_id` int(11) NOT NULL DEFAULT '0',
  `p_id` int(11) NOT NULL DEFAULT '0',
  `rolename` varchar(255) NOT NULL DEFAULT '',
  PRIMARY KEY  (`role_id`),
  constraint `user_role_id` FOREIGN KEY (`p_id`) REFERENCES `test_user` (`p_id`)
) engine=innodb DEFAULT charset=utf8;

CREATE TABLE `test_obs` (
  `obs_id` int(11) NOT NULL DEFAULT '0',
  `p_id` int(11) NOT NULL DEFAULT '0',
  `observation` varchar(255) NOT NULL DEFAULT '',
  PRIMARY KEY  (`obs_id`),
  constraint `patient_obs_id` FOREIGN KEY (`p_id`) REFERENCES `test_patient` (`p_id`)
) engine=innodb DEFAULT charset=utf8;

INSERT INTO test_person (p_id, name) VALUES (1, 'Christian');
INSERT INTO test_person (p_id, name) VALUES (2, 'Darius');
INSERT INTO test_person (p_id, name) VALUES (3, 'Justin');
INSERT INTO test_person (p_id, name) VALUES (4, 'Aunt Jemima');
 
SELECT * FROM test_person;
 
INSERT INTO test_patient(p_id, location) VALUES(1, 'rwinkwavu hospital');
INSERT INTO test_patient(p_id, location) VALUES(4, 'supermarket shelves everywhere');
INSERT INTO test_patient(p_id, location) VALUES(5, 'this wont work');
 
SELECT * FROM test_patient;
 
INSERT INTO test_user(p_id, username) VALUES(1, 'callen');
INSERT INTO test_user(p_id, username) VALUES(3, 'jmiranda');
INSERT INTO test_user(p_id, username) VALUES(5, 'thiswontwork');
 
SELECT * FROM test_user;
 
INSERT INTO test_role(role_id, p_id, rolename) VALUES(1, 1, 'village idiot');
INSERT INTO test_role(role_id, p_id, rolename) VALUES(2, 2, 'darius is just a person so this should not work');
INSERT INTO test_role(role_id, p_id, rolename) VALUES(3, 3, 'van triliquist');
INSERT INTO test_role(role_id, p_id, rolename) VALUES(4, 4, 'aunt jemima is a patient but not a user so this should not work');
INSERT INTO test_role(role_id, p_id, rolename) VALUES(5, 5, 'thiswontwork');
 
SELECT * FROM test_role;
 
INSERT INTO test_obs(obs_id, p_id, observation) VALUES(1, 1, 'needs a better haircut');
INSERT INTO test_obs(obs_id, p_id, observation) VALUES(2, 3, 'justin is only a user so this should not work');
INSERT INTO test_obs(obs_id, p_id, observation) VALUES(3, 4, 'loves monorails');
INSERT INTO test_obs(obs_id, p_id, observation) VALUES(4, 5, 'this wont work');
 
SELECT * FROM test_obs;
 
ALTER TABLE test_obs DROP FOREIGN KEY patient_obs_id;
ALTER