aboutsummaryrefslogtreecommitdiffstats
path: root/mail/README.async
diff options
context:
space:
mode:
Diffstat (limited to 'mail/README.async')
-rw-r--r--mail/README.async360
1 files changed, 0 insertions, 360 deletions
diff --git a/mail/README.async b/mail/README.async
deleted file mode 100644
index 238fafcedb..0000000000
--- a/mail/README.async
+++ /dev/null
@@ -1,360 +0,0 @@
-Asynchronous Mailer Information
-Peter Williams <peterw@helixcode.com>
-8/4/2000
-
-1. INTRODUCTION
-
-It's pretty clear that the Evolution mailer needs to be asynchronous in
-some manner. Blocking the UI completely on IMAP calls or large disk reads
-is unnacceptable, and it's really nice to be able to thread the message
-view in the background, or do other things while a mailbox is downloading.
-
-The problem in making Evolution asynchronous is Camel. Camel is not
-asynchronous for a few reasons. All of its interfaces are synchronous --
-calls like camel_store_get_folder, camel_folder_get_message, etc. can
-take a very long time if they're being performed over a network or with
-a large mbox mailbox file. However, these functions have no mechanism
-for specifying that the operation is in progress but not complete, and
-no mechanism for signaling when to operation does complete.
-
-2. WHY I DIDN'T MAKE CAMEL ASYNCHRONOUS
-
-It seems like it would be a good idea, then, to rewrite Camel to be
-asynchonous. This presents several problems:
-
- * Many interfaces must be rewritten to support "completed"
- callbacks, etc. Some of these interfaces are connected to
- synchronous CORBA calls.
- * Everything must be rewritten to be asynchonous. This includes
- the CamelStore, CamelFolder, CamelMimeMessage, CamelProvider,
- every subclass thereof, and all the code that touches these.
- These include the files in camel/, mail/, filter/, and
- composer/. The change would be a complete redesign for any
- provider implementation.
- * All the work on providers (IMAP, mh, mbox, nntp) up to this
- point would be wasted. While they were being rewritten
- evolution-mail would be useless.
-
-However, it is worth noting that the solution I chose is not optimal,
-and I think that it would be worthwhile to write a libcamel2 or some
-such thing that was designed from the ground up to work asynchronously.
-Starting fresh from such a design would work, but trying to move the
-existing code over would be more trouble than it's worth.
-
-3. WHY I MADE CAMEL THREADED
-
-If Camel was not going to be made asynchronous, really the only other
-choice was to make it multithreaded. [1] No one has been particularly
-excited by this plan, as debugging and writing MT-safe code is hard.
-But there wasn't much of a choice, and I believed that a simple thread
-wrapper could be written around Camel.
-
-The important thing to know is that while Camel is multithreaded, we
-DO NOT and CANNOT share objects between threads. Instead,
-evolution-mail sends a request to a dispatching thread, which performs
-the action or queues it to be performed. (See section 4 for details)
-
-The goal that I was working towards is that there should be no calls
-to camel made, ever, in the main thread. I didn't expect to and
-didn't do this, but that was the intent.
-
-[1]. Well, we could fork off another process, but they share so much
-data that this would be pretty impractical.
-
-4. IMPLEMENTATION
-
-a. CamelObject
-
-Threading presented a major problem regarding Camel. Camel is based
-on the GTK Object system, and uses signals to communicate events. This
-is okay in a nonthreaded application, but the GTK Object system is
-not thread-safe.
-
-Particularly, signals and object allocations use static data. Using
-either one inside Camel would guarantee that we'd be stuck with
-random crashes forevermore. That's Bad (TM).
-
-There were two choices: make sure to limit our usage of GTK, or stop
-using the GTK Object system. I decided to do the latter, as the
-former would lead to a mess of "what GTK calls can we make" and
-GDK_THREADS_ENTER and accidentally calling UI functions and upgrades
-to GTK breaking everything.
-
-So I wrote a very very simple CamelObject system. It had three goals:
-
- * Be really straightforward, just encapsulate the type
- heirarchy without all that GtkArg silliness or anything.
- * Be as compatible as possible with the GTK Object system
- to make porting easy
- * Be threadsafe
-
-It supports:
-
- * Type inheritance
- * Events (signals)
- * Type checking
- * Normal refcounting
- * No unref/destroy messiness
- * Threadsafety
- * Class functions
-
-The entire code to the object system is in camel/camel-object.c. It's
-a naive implementation and not full of features, but intentionally that
-way. The main differences between GTK Objects and Camel Objects are:
-
- * s,gtk,camel,i of course
- * Finalize is no longer a GtkObjectClass function. You specify
- a finalize function along with an init function when declaring
- a type, and it is called automatically and chained automatically.
- * Declaring a type is a slightly different API
- * The signal system is replaced with a not-so-clever event system.
- Every event is equivalent to a NONE__POINTER signal. The syntax
- is slightly different: a class "declares" an event and specifies
- a name and a "prep func", that is called before the event is
- triggered and can cancel it.
- * There is only one CamelXXXClass in existence for every type.
- All objects share it.
-
-There is a shell script, tools/make-camel-object.sh that will do all of
-the common substitutions to make a file CamelObject-compatible. Usually
-all that needs to be done is move the implementation of the finalize
-event out of the class init, modify the get_type function, and replace
-signals with events.
-
-Pitfalls in the transition that I ran into were:
-
- * gtk_object_ref -> camel_object_ref or you coredump
- * some files return 'guint' instead of GtkType and must be changed
- * Remove the #include <gtk/gtk.h>
- * gtk_object_set_datas must be changed (This happened once; I
- added a static hashtable)
- * signals have to be fudged a bit to match the gpointer input
- * the BAST_CASTARD option is on, meaning failed typecasts will
- return NULL, almost guaranteeing a segfault -- gets those
- bugs fixed double-quick!
-
-b. API -- mail_operation_spec
-
-I worked by creating a very specific definition of a "mail operation"
-and wrote an engine to queue and dispatch them.
-
-A mail operation is defined by a structure mail_operation_spec
-prototyped in mail-threads.h. It comes in three logical parts -- a
-"setup" phase, executed in the main thread; a "do" phase, executed
-in the dispatch thread; and a "cleanup" phase, executed in the main
-thread. These three phases are guaranteed to be performed in order
-and atomically with respect to other mail operations.
-
-Each of these phases is represented by a function pointer in the
-mail_operation_spec structure. The function mail_operation_queue() is
-called and passed a pointer to a mail_operation_spec and a user_data-style
-pointer that fills in the operation's parameters. The "setup" callback
-is called immediately, though that may change.
-
-Each callback is passed three parameters: a pointer to the user_data,
-a pointer to the "operation data", and a pointer to a CamelException.
-The "operation data" is allocated automatically and freed when the operation
-completes. Internal data that needs to be shared between phases should
-be stored here. The size allocated is specified in the mail_operation_spec
-structure.
-
-Because all of the callbacks use Camel calls at some point, the
-CamelException is provided as utility. The dispatcher will catch exceptions
-and display error dialogs, unlike the synchronous code which lets
-exceptions fall through the cracks fairly easily.
-
-I tried to implement all the operations following this convention. Basically
-I used this skeleton code for all the operations, just filling in the
-specifics:
-
-===================================
-
-typedef struct operation_name_input_s {
- parameters to operation
-} operation_name_input_t;
-
-typedef struct operation_name_data_s {
- internal data to operation, if any
- (if none, omit the structure and set opdata_size to 0)
-} operation_name_data_t;
-
-static gchar *describe_operation_name (gpointer in_data, gboolean gerund);
-static void setup_operation_name (gpointer in_data, gpointer op_data, CamelException *ex);
-static void do_operation_name (gpointer in_data, gpointer op_data, CamelException *ex);
-static void cleanup_operation_name (gpointer in_data, gpointer op_data, CamelException *ex);
-
-static gchar *describe_operation_name (gpointer in_data, gboolean gerund)
-{
- operation_name_input_t *input = (operation_name_input_t *) in_data;
-
- if (gerund) {
- return a g_strdup'ed string describing what we're doing
- } else {
- return a g_strdup'ed string describing what we're about to do
- }
-}
-
-static void setup_operation_name (gpointer in_data, gpointer op_data, CamelException *ex)
-{
- operation_name_input_t *input = (operation_name_input_t *) in_data;
- operation_name_data_t *data = (operation_name_data_t *) op_data;
-
- verify that parameters are valid
-
- initialize op_data
-
- reference objects
-}
-
-static void do_operation_name (gpointer in_data, gpointer op_data, CamelException *ex)
-{
- operation_name_input_t *input = (operation_name_input_t *) in_data;
- operation_name_data_t *data = (operation_name_data_t *) op_data;
-
- perform camel operations
-}
-
-static void cleanup_operation_name (gpointer in_data, gpointer op_data, CamelException *ex)
-{
- operation_name_input_t *input = (operation_name_input_t *) in_data;
- operation_name_data_t *data = (operation_name_data_t *) op_data;
-
- perform UI updates
-
- free allocations
-
- dereference objects
-}
-
-static const mail_operation_spec op_operation_name = {
- describe_operation_name,
- sizeof (operation_name_data_t),
- setup_operation_name,
- do_operation_name,
- cleanup_operation_name
-};
-
-void
-mail_do_operation_name (parameters)
-{
- operation_name_input_t *input;
-
- input = g_new (operation_name_input_t, 1);
-
- store parameters in input
-
- mail_operation_queue (&op_operation_name, input, TRUE);
-}
-
-===========================================
-
-c. mail-ops.c
-
-Has been drawn and quartered. It has been split into:
-
- * mail-callbacks.c: the UI callbacks
- * mail-tools.c: useful sequences wrapping common Camel operations
- * mail-ops.c: implementations of all the mail_operation_specs
-
-An important part of mail-ops.c are the global functions
-mail_tool_camel_lock_{up,down}. These simulate a recursize mutex around
-camel. There are an extreme few, supposedly safe, calls to Camel made in
-the main thread. These functions should go around evey call to Camel or
-group thereof. I don't think they're necessary but it's nice to know
-they're there.
-
-If you look at mail-tools.c, you'll notice that all the Camel calls are
-protected with these functions. Remember that a mail tool is really
-just another Camel call, so don't use them in the main thread either.
-
-All the mail operations are implemented in mail-ops.c EXCEPT:
-
- * filter-driver.c: the filter_mail operation
- * message-list.c: the regenerate_messagelist operation
- * message-thread.c: the thread_messages operation
-
-d. Using the operations
-
-The mail operations as implemented are very specific to evolution-mail. I
-was thinking about leaving them mostly generic and then allowing extra
-callbacks to be added to perform the more specific UI touches, but this
-seemed kind of pointless.
-
-I basically looked through the code, found references to Camel, and split
-the code into three parts -- the bit before the Camel calls, the bit after,
-and the Camel calls. These were mapped onto the template, given a name,
-and added to mail-ops.c. Additionally, I simplified the common tasks that
-were taken care of in mail-tools.c, making some functions much simpler.
-
-Ninety-nine percent of the time, whatever operation is being done is being
-done in a callback, so all that has to be done is this:
-
-==================
-
-void my_callback (GtkObject *obj, gchar *uid)
-{
- camel_do_something (uid);
-}
-
-==== becomes ====
-
-void my_callback (GtkObject *obj, gchar *uid)
-{
- mail_do_do_something (uid);
-}
-
-=================
-
-There are, however, a few belligerents. Particularly, the function
-mail_uri_to_folder returns a CamelFolder and yet should really be
-asynchronous. This is called in a CORBA call that is sychronous, and
-additionally is used in the filter code.
-
-I changed the first usage to return the folder immediately but
-still fetch the CamelFolder asyncrhonously, and in the second case,
-made filtering asynchronous, so the fact that the call is synchronous
-doesn't matter.
-
-The function was renamed to mail_tool_uri_to_folder to emphasize that
-it's a synchronous Camel call.
-
-e. The dispatcher
-
-mail_operation_queue () takes its parameters and assembles them in a
-closure_t structure, which I abbreviate clur. It sets a timeout to
-display a progress window if an operation is still running one second
-later (we're not smart enough to check if it's the same operation,
-but the issue is not a big deal). The other thread and some communication
-pipes are created.
-
-The dispatcher thread sits in a loop reading from a pipe. Every time
-the main thread queues an operation, it writes the closure_t into the pipe.
-The dispatcher reads the closure, sends a STARTING message to the main
-thread (see below for explanation), calls the callback specified in the
-closure, and sends a FINISHED message. It then goes back to reading
-from its pipe; it will either block until another operation comes along,
-or find one right away and start it. This the pipe takes care of queueing
-operations.
-
-The dispatch thread communicates with the main thread with another pipe;
-however, the main thread has other things to do than read from the pipe,
-so it adds registers a GIOReader that checks for messages in the glib
-main loop. In addition to starting and finishing messages, the other
-thread can communicate to the user using messages and a progress bar.
-(This is currently implemented but unused.)
-
-5. ISSUES
-
- * Operations are queued and dequeued stupidly. Like if you click
- on one message then click on another, the first will be retrieved
- and displayed then overwritten by the second. Operations that could
- be performed at the same time safely aren't.
- * The CamelObject system is workable, but it'd be nice to work with
- something established like the GtkObject
- * The whole threading idea is not great. Concensus is that an
- asynchronous interface is the Right Thing, eventually.
- * Care still needs to be taken when designing evolution-mail code to
- work with the asynchronous mail_do_ functions
- * Some of the operations are extremely hacky.
- * IMAP's timeout to send a NOOP had to be removed because we can't
- use GTK. We need an alternative for this. \ No newline at end of file