A really essential element in any kind of personal information manager is a recurrence pattern.  You know: “every 3 days”; “at the beginning of the month”; “2 weeks after I’m done.”

For some kinds of PIM apps — habit managers come to mind — you might even say that the richness of the recurrence pattern repertoire is the main element driving me to pick one over the other.

The problem is that doing recurrence patterns is hard, and doing them right is even harder.  They are perverse.

Recurrence patterns inherit at least some of their perversity from the calendar itself.  We have years that don’t divide into weeks.  We have months that don’t divide into weeks either, and that don’t even have a uniform length.  We have weekdays and weekends.  We even have different clocks, since our legacy clock the Earth is slowing down.

(Scientists add a second to the year every few years.  I have this picture of a team in white coats bringing over a bar of light a light-second long and opening an access cover and throwing into the stream of light that is Time.  Given how long a light-second is, that’s a pretty big team.)

But then the nature of recurrence adds some extra perversity all its own.

In the first place, some recurrences are regular and some are driven by completion of the event.  “Every two days” is not the same as “two days after I finish.”

And some recurrences have more oomph (a technical term!) than others: you’ve got to take your medicine every day, you ought to exercise every day (but the sky won’t fall if you don’t), and it would be nice to write a handwritten thank-you note after every dinner party.

(Perhaps the oomph is best handled not in the recurrence mechanism itself but in a orthogonal priority or urgency mechanism.  Even so, it would be nice to express the oomph at the same time you created the recurrence pattern (but see Syntax and Command Language below).)

And then, as if the foregoing weren’t enough, some recurrences are fuzzy.  “Every 2-3 days.”  “4 times a week max.”

I once used a habit manager app for a few months just because it had some cool fuzzy recurrence features.  It didn’t interoperate with anything else, it only ran on iOS, and it didn’t sync with the cloud — all flaws that eventually drove it down.  But those fuzzy recurrences were pretty cool: I miss them still.

So these are variety issues.  Recurrence patterns are hard in part because of the variety.

But there are (at least?) three issues with recurrence patterns aside from their variety:

1. Controller interface.  A lot of recurrence providers try to offer a natural-language interface to recurrence patterns.  “Just type in a phrase that describes when your task has to be done”.    I’m not blown away by these (and I’m an old NLP guy from the ’80s).  You really can’t say what you mean often: “The 2nd Saturday of each month” may well parse to “every other Saturday” in one system and to its intended meaning in another.  Having a form to fill out is better, because you can at least see all the options that can be generated, but forms don’t do well with fuzzy recurrences.
2. Internal Representation.  Whatever the implementation of a task or an event, a recurring task or event is a different kind of animal: it’s not a data structure, it’s an object.  Outlook, for example, doesn’t even seem to store recurring objects in the same file as task or calendar objects.  When you migrate, they don’t always migrate together.
3. “Round Trip”.  The different representations of recurring objects means that at some point they will need to “compile” to their non-recurring cousins.  One would want this to be a round trip in the sense that you could traverse from the recurrence to the end objects and also reconstitute the recurrence from the end objects.  No one does this.  Once a recurring object has been instantiated, there’s no going back.

For all these reasons, recurring patterns are hard, and getting them right is an architectural key to a great PIM.

Part of building a better PIM infrastructure is undoubtedly solving the “tagging problem.”

The tagging problem is: how do you get users of a PIM system to tag their data 1) promptly 2) accurately 3) with regard to downstream users, and 4) MECE-ly without requiring them to use a heavy-duty ontology of some sort?

1. Promptly.  The best thing is to tag data at the point of production, at least for salience to the person entering the data.  For downstream taggers, promptness is also of value as the information comes to them.
2. Accurately.  There are two senses of accurate (maybe more?): Accuracy1 is making sure that data with the same semantics gets the same tags.  Accuracy2 is making sure that the tag is appropriate to the data.
3. Helpful Downstream.  In a perfect world, the tagging with all be helpful to future users of the data.
4. MECE-ly.  MECE is McKinsey’s acronym for “mutually exclusive, collectively exhaustive” which describes a set of tags which, for want of a better word, are what linear algebra would call a basis for a knowledge space.  Every item fits into one of the tags, and no data fits into two of the tags.

The tagging problem has no real solution, and the lack of a solution may be the downfall of many a knowledge system, whether personal or tribe.

Solutions are elusive because it’s not immediately in the interests of the tagger to do the tagging, even when it might be in their long-term interest.  Of course, it’s always in the long-term interest of any user of the system.

But what could you do if tagging occurred as an automatic by-product of some other operation?  One that users wanted to do?

I once trained an open-source Bayesian categorizer to distinguish spam from bacn from desirable mail in my inbox.  I did it by doing what I would have done anyhow — dragging the incoming mail into good, bacn, and spam folders.  But while I was doing that the classifier was changing its weights (or whatever classifiers do) and automatically improving its ability to triage my mail.

Could that same technique be applied to more complicated tagging problems?  Not sure.  It needs some thought.

First of all, I’m not a MIcrosoft basher.

I’m not even a big foe of Microsoft.  I made a good living from programming with Microsoft products for years in the ’80’s and ’90’s, and I don’t like to bite the hand that fed me.

But, more than that, there are Microsoft tools I’ve liked, some I’ve loved, and some I still love (Excel, yes, and Word too).

However, Microsoft search, almost across the entire company, is terrible.  Bing search is inferior.  Visual Studio search is impossible.  Desktop search didn’t work well at all.  Windows 8/10 search is very very lame.

With one exception, perhaps: as I wrote here, I’ve moved off of Evernote back to OneNote, and OneNote search is pretty darn good.

First of all, it’s better than Evernote search.

Evernote search is quirky and clunky.  I type in a word that’s in the text and in a tag, and then when I click on the tag (thinking I’ll get all the results with that tag) I inexplicably get nothing at all.  I have to go back and say “search by tag” and then scroll down and find the tag and click on it.  I don’t get it.

In OneNote, you type in a word, you get a list of pages grouped by salience (at the top are pages with the search term in a title, then in the body, then in less-visited places (Recycle bin, Misplaced sections, etc.).  You click on a search result and you go to that page.  Take a note from OneNote, guys at Evernote.  It doesn’t have to be clunky.

I regularly feel the bite of too many PIM tools… and not enough PIM interoperability.

Ferinstance, right now…

I’m using MLO as my core To-do tool with some satisfaction so far, as discussed here.

No to-do tool really does a good job with tracking Jerry Seinfeld’s “Don’t Break the Chain” stuff (mainly because they lack a rich set of recurrence patterns, stuff like “every 2-3 days” or “at least 4 times a week”), so I use an Android app, HabitBull, for that: decent, nice visuals, wish it were cross-platform.

(BTW, I generally get the Pro [pay] versions of these apps; first of all, they generally have features I like, but, more importantly, I wrote software for a living for 20+ years; I want the people who write these apps to make a living from it too.)

Now I’ve started messing around with Jane McGonigal’s “SuperBetter” methodology and app.  I got interested in her stuff through Tim Ferriss’ blog on her.  Her shtick is that you gain on problems by using know-how from gaming.   Not gamification (badges and other superficial rewards), which is dorky, but what she calls a “gameful” approach to problems.  I’m a little embarrassed about the game vocabulary and gloss — “power-ups”, “bad guys”, “quests” — but not really embarrassed enough not to try it.  And my attitude has always been, like Jurgen the pawnbroker, “I’ll take any drink once.”

Finally, I’ve been intrigued by the bullet journal, which is a compact diary with some elements of life-tracking.

So, MLO, HabitBull, SuperBetter, Bullet journal (in OneNote).  Four systems to keep in sync, to keep up-to-date.  What a pain (although, of course a First-World pain.)

Why can’t these platforms interoperate?

Two problems, one lesser, one greater:

Lesser Problem: they hang onto their own data.  As we used to say back in the PC revolution of the ’80’s, they all have an Import but none of them has an Export.  Or not a very good Export.

Greater Problem: It still takes a Master Ontology to comprise all of them, and keeping a Master Ontology up-to-date is a nightmare.

Wouldn’t it be nice if they could interoperate without an ontology, or with a very light one?  Stay tuned.

I had breakfast with Larry this morning, and, among other topics, we discussed two PIM concepts he had raised in an email: “swim lanes” and “threads”.

Swim lanes are a familiar Kanban concept: you have a Kanban board divided into columns showing steps in a work process: “To Do”, “Doing”, “Done” in the base case.  Swim lanes divide the columns horizontally:

For a project board like this one, swim lanes are something like modules within the product or project.  For a personal board, such as we were talking about for PIM, swim lanes seem closes to “Roles” — “Parent”, “Manager”, “Pilgrim”, etc.

The virtue of swim lanes, per Larry, is that they show, at a glance, how the pending workflow is divided by role.  Are you spending too much time on work stuff, are you shorting your spiritual development.  The column limit makes sure you aren’t taking on too much WIP at once.  The horizontal swim lane shows how things are going in the role.

“Threads” turned out to be a subtler idea, and one that neither Larry’s Kanban world nor my “hierarchy of tasks” world does particularly well.

Per Larry, a thread is, like in computers, a lightweight process where the sequence of tasks is important and the generation of new tasks is important.

We all have experiences like this, where what seems like a simple atomic task turns out to have subparts, and where the thread itself generates new tasks as it goes along.

Larry’s example was spec-ing windows for a house he’s building.  The process of vetting each particular window vendor spawns new tasks, and the process itself generates a need to vet new vendors.

Larry wanted some element on the Kanban board that visually tracks this “thread” relationship.

I guess I handle this in my hierarchy world by an exploded view of the thread.  By just exploding the thread itself, but keeping the rest of the hierarchy collapsed, you can see the relationships between the tasks and the new tasks in the thread:

Larry wasn’t completely happy with this, but it was a start.

How to handle swim lanes/roles and threads?  Your comments?