A really essential element in any kind of personal information manager is a recurrence pattern.  You know: “every 3 days”; “at the beginning of the month”; “2 weeks after I’m done.”

For some kinds of PIM apps — habit managers come to mind — you might even say that the richness of the recurrence pattern repertoire is the main element driving me to pick one over the other.

The problem is that doing recurrence patterns is hard, and doing them right is even harder.  They are perverse.

Recurrence patterns inherit at least some of their perversity from the calendar itself.  We have years that don’t divide into weeks.  We have months that don’t divide into weeks either, and that don’t even have a uniform length.  We have weekdays and weekends.  We even have different clocks, since our legacy clock the Earth is slowing down.

(Scientists add a second to the year every few years.  I have this picture of a team in white coats bringing over a bar of light a light-second long and opening an access cover and throwing into the stream of light that is Time.  Given how long a light-second is, that’s a pretty big team.)

But then the nature of recurrence adds some extra perversity all its own.

In the first place, some recurrences are regular and some are driven by completion of the event.  “Every two days” is not the same as “two days after I finish.”

And some recurrences have more oomph (a technical term!) than others: you’ve got to take your medicine every day, you ought to exercise every day (but the sky won’t fall if you don’t), and it would be nice to write a handwritten thank-you note after every dinner party.

(Perhaps the oomph is best handled not in the recurrence mechanism itself but in a orthogonal priority or urgency mechanism.  Even so, it would be nice to express the oomph at the same time you created the recurrence pattern (but see Syntax and Command Language below).)

And then, as if the foregoing weren’t enough, some recurrences are fuzzy.  “Every 2-3 days.”  “4 times a week max.”

I once used a habit manager app for a few months just because it had some cool fuzzy recurrence features.  It didn’t interoperate with anything else, it only ran on iOS, and it didn’t sync with the cloud — all flaws that eventually drove it down.  But those fuzzy recurrences were pretty cool: I miss them still.

So these are variety issues.  Recurrence patterns are hard in part because of the variety.

But there are (at least?) three issues with recurrence patterns aside from their variety:

1. Controller interface.  A lot of recurrence providers try to offer a natural-language interface to recurrence patterns.  “Just type in a phrase that describes when your task has to be done”.    I’m not blown away by these (and I’m an old NLP guy from the ’80s).  You really can’t say what you mean often: “The 2nd Saturday of each month” may well parse to “every other Saturday” in one system and to its intended meaning in another.  Having a form to fill out is better, because you can at least see all the options that can be generated, but forms don’t do well with fuzzy recurrences.
2. Internal Representation.  Whatever the implementation of a task or an event, a recurring task or event is a different kind of animal: it’s not a data structure, it’s an object.  Outlook, for example, doesn’t even seem to store recurring objects in the same file as task or calendar objects.  When you migrate, they don’t always migrate together.
3. “Round Trip”.  The different representations of recurring objects means that at some point they will need to “compile” to their non-recurring cousins.  One would want this to be a round trip in the sense that you could traverse from the recurrence to the end objects and also reconstitute the recurrence from the end objects.  No one does this.  Once a recurring object has been instantiated, there’s no going back.

For all these reasons, recurring patterns are hard, and getting them right is an architectural key to a great PIM.

Part of building a better PIM infrastructure is undoubtedly solving the “tagging problem.”

The tagging problem is: how do you get users of a PIM system to tag their data 1) promptly 2) accurately 3) with regard to downstream users, and 4) MECE-ly without requiring them to use a heavy-duty ontology of some sort?

1. Promptly.  The best thing is to tag data at the point of production, at least for salience to the person entering the data.  For downstream taggers, promptness is also of value as the information comes to them.
2. Accurately.  There are two senses of accurate (maybe more?): Accuracy1 is making sure that data with the same semantics gets the same tags.  Accuracy2 is making sure that the tag is appropriate to the data.
3. Helpful Downstream.  In a perfect world, the tagging with all be helpful to future users of the data.
4. MECE-ly.  MECE is McKinsey’s acronym for “mutually exclusive, collectively exhaustive” which describes a set of tags which, for want of a better word, are what linear algebra would call a basis for a knowledge space.  Every item fits into one of the tags, and no data fits into two of the tags.

The tagging problem has no real solution, and the lack of a solution may be the downfall of many a knowledge system, whether personal or tribe.

Solutions are elusive because it’s not immediately in the interests of the tagger to do the tagging, even when it might be in their long-term interest.  Of course, it’s always in the long-term interest of any user of the system.

But what could you do if tagging occurred as an automatic by-product of some other operation?  One that users wanted to do?

I once trained an open-source Bayesian categorizer to distinguish spam from bacn from desirable mail in my inbox.  I did it by doing what I would have done anyhow — dragging the incoming mail into good, bacn, and spam folders.  But while I was doing that the classifier was changing its weights (or whatever classifiers do) and automatically improving its ability to triage my mail.

Could that same technique be applied to more complicated tagging problems?  Not sure.  It needs some thought.

First of all, I’m not a MIcrosoft basher.

I’m not even a big foe of Microsoft.  I made a good living from programming with Microsoft products for years in the ’80’s and ’90’s, and I don’t like to bite the hand that fed me.

But, more than that, there are Microsoft tools I’ve liked, some I’ve loved, and some I still love (Excel, yes, and Word too).

However, Microsoft search, almost across the entire company, is terrible.  Bing search is inferior.  Visual Studio search is impossible.  Desktop search didn’t work well at all.  Windows 8/10 search is very very lame.

With one exception, perhaps: as I wrote here, I’ve moved off of Evernote back to OneNote, and OneNote search is pretty darn good.

First of all, it’s better than Evernote search.

Evernote search is quirky and clunky.  I type in a word that’s in the text and in a tag, and then when I click on the tag (thinking I’ll get all the results with that tag) I inexplicably get nothing at all.  I have to go back and say “search by tag” and then scroll down and find the tag and click on it.  I don’t get it.

In OneNote, you type in a word, you get a list of pages grouped by salience (at the top are pages with the search term in a title, then in the body, then in less-visited places (Recycle bin, Misplaced sections, etc.).  You click on a search result and you go to that page.  Take a note from OneNote, guys at Evernote.  It doesn’t have to be clunky.

I regularly feel the bite of too many PIM tools… and not enough PIM interoperability.

Ferinstance, right now…

I’m using MLO as my core To-do tool with some satisfaction so far, as discussed here.

No to-do tool really does a good job with tracking Jerry Seinfeld’s “Don’t Break the Chain” stuff (mainly because they lack a rich set of recurrence patterns, stuff like “every 2-3 days” or “at least 4 times a week”), so I use an Android app, HabitBull, for that: decent, nice visuals, wish it were cross-platform.

(BTW, I generally get the Pro [pay] versions of these apps; first of all, they generally have features I like, but, more importantly, I wrote software for a living for 20+ years; I want the people who write these apps to make a living from it too.)

Now I’ve started messing around with Jane McGonigal’s “SuperBetter” methodology and app.  I got interested in her stuff through Tim Ferriss’ blog on her.  Her shtick is that you gain on problems by using know-how from gaming.   Not gamification (badges and other superficial rewards), which is dorky, but what she calls a “gameful” approach to problems.  I’m a little embarrassed about the game vocabulary and gloss — “power-ups”, “bad guys”, “quests” — but not really embarrassed enough not to try it.  And my attitude has always been, like Jurgen the pawnbroker, “I’ll take any drink once.”

Finally, I’ve been intrigued by the bullet journal, which is a compact diary with some elements of life-tracking.

So, MLO, HabitBull, SuperBetter, Bullet journal (in OneNote).  Four systems to keep in sync, to keep up-to-date.  What a pain (although, of course a First-World pain.)

Why can’t these platforms interoperate?

Two problems, one lesser, one greater:

Lesser Problem: they hang onto their own data.  As we used to say back in the PC revolution of the ’80’s, they all have an Import but none of them has an Export.  Or not a very good Export.

Greater Problem: It still takes a Master Ontology to comprise all of them, and keeping a Master Ontology up-to-date is a nightmare.

Wouldn’t it be nice if they could interoperate without an ontology, or with a very light one?  Stay tuned.

I had breakfast with Larry this morning, and, among other topics, we discussed two PIM concepts he had raised in an email: “swim lanes” and “threads”.

Swim lanes are a familiar Kanban concept: you have a Kanban board divided into columns showing steps in a work process: “To Do”, “Doing”, “Done” in the base case.  Swim lanes divide the columns horizontally:

For a project board like this one, swim lanes are something like modules within the product or project.  For a personal board, such as we were talking about for PIM, swim lanes seem closes to “Roles” — “Parent”, “Manager”, “Pilgrim”, etc.

The virtue of swim lanes, per Larry, is that they show, at a glance, how the pending workflow is divided by role.  Are you spending too much time on work stuff, are you shorting your spiritual development.  The column limit makes sure you aren’t taking on too much WIP at once.  The horizontal swim lane shows how things are going in the role.

“Threads” turned out to be a subtler idea, and one that neither Larry’s Kanban world nor my “hierarchy of tasks” world does particularly well.

Per Larry, a thread is, like in computers, a lightweight process where the sequence of tasks is important and the generation of new tasks is important.

We all have experiences like this, where what seems like a simple atomic task turns out to have subparts, and where the thread itself generates new tasks as it goes along.

Larry’s example was spec-ing windows for a house he’s building.  The process of vetting each particular window vendor spawns new tasks, and the process itself generates a need to vet new vendors.

Larry wanted some element on the Kanban board that visually tracks this “thread” relationship.

I guess I handle this in my hierarchy world by an exploded view of the thread.  By just exploding the thread itself, but keeping the rest of the hierarchy collapsed, you can see the relationships between the tasks and the new tasks in the thread:

Larry wasn’t completely happy with this, but it was a start.

How to handle swim lanes/roles and threads?  Your comments?

I’ve been pretty fond of Evernote for a while.

I was a very early adopter of OneNote (and loved it too, despite its heavy client ways).  What switched me over to Evernote was a Todo list app called ZenDone with some very cute ideas, among which was deep integration with Evernote.

Almost all the data for a todo item in ZenDone was an Evernote note, and the notebooks themselves were used as GTD lists.

I ultimately left ZenDone because its performance was marginal too often and the company was in denial that I had a problem.

(And the deep integration with Evernote, btw, made it a bear to unwind and export all of the data.)

But the whole experience had gotten me kind of jazzed about Evernote, mainly because it looked pretty nice in almost all the clients and because it had a client for every environment where I either needed to produce or consume notes.  Universal coverage is pretty compelling.

So I simulated the ZenDone integration, first with ToodleDo, then with Todoist, and now with My Life Organized.  Bit of a kludge in all cases, (ZenDone just kind of did the integration for you), but still useful.

And then this morning I lost data on two of my Evernote notes.

They were produced by scanned-in magazine pages vis Scansnap, which produces a kind of embedded pdf within the Evernote note.

I clicked on the button to produce a shared link for the two notes, and the embedded pdf vanished.  Nothing left but the note subject.

I had thrown out the magazine pages once scanned, so there was no going back to them.

I looked on the web and found an idea of a note history.  I was able to roll back the history of both notes to restore the embedded content.  But then, of course, I had to try generating the shared link again, and the one note I tried it on promptly liquidated the content again, and this time I couldn’t get it back.

I had read a fair amount of stuff on the web about how marginal the Evernote software was, and, indeed, I have experienced ceaseless crashes of the clients (and even the web client) on various platforms.

But I had never lost data before.

OK.  Enough.  I had a look at the state of play of OneNote this morning, and I’m going to move over.  The gmail client for OneNote is nothing like as cool as the Evernote one, but there’s good platform coverage and it looks like I can get my work done with no giant gotchas.

Sorry, Evernote.  Maybe another time.

According to GTD — IMHO — there are two reasons to have a to-do list manager:

1. Keep track of every “loose end” in the GTD sense of “thing that would otherwise bug me if it weren’t recorded in a trusted system”.
2. Supply a palette of small-bite “next actions” which can be selected when one has a question of what to do next.

Most to-do list systems manage these two pretty well.  Where they begin to separate (again, IMHO) is how they handle a third problem: how to pick what to do next.

When fitzpatl@gmail.com comments that “[MLO] could get complicated pretty fast”, he is coming from the Kanban place of “focus on a few things by limited next actions-in-progress”.    There is power in focus — ultimately, this ends up as the kind of focus Gary Keller celebrates in “The One Thing”: a singleton.

I think about The One Thing a lot, but still there’s something in me that wants to see all the next actions, and, even more, there’s something in me that wants software to automate the ranking of the next actions.

I’m not sure this is in the service of productivity anymore.  It might be a kind of Bertrand-Russell-like fascination with reifying my entire universe of effort into an automated list.  But I think I’m not the only person who flirts with this kind of thing.

There’s actually to-do-list software I used for a while a long time ago that automatically fills out your calendar for you with tasks.  You tell it the priorities of tasks and their duration; you tell it your appointments and your work-stop and work-start times, and it will fill the calendar with scheduled, prioritized tasks.

I loved that hack while I using it.  I couldn’t really use it because it was too cumbersome.  And I abandoned it although I loved it.

I’m really interested in Personal Information Management (PIM) tools, and I want to write about them some in coming  posts.

People in the Real World who know me know two things about me and personal information management (let’s just agree right now to call it PIM):

1. I’m passionate about the theory and practice of keeping personal information — such as goals and dreams, but also such as steps and calories — and using it to hopefully make myself a better person.
2. I am a fickle user.  I’ve been through 15 or 20 To-do list managers in the years I’ve been doing this.  If a new one catches my fancy, I’m not afraid to put in a lot of work to move from Old to New, and usually right away.

Until two weeks ago I was pretty happy with Todoist, a cloud-based app with some very nice features.

It ran on everything I’ve got — two desktop PCs, a MacBook Pro, two iPads, an Android phone, and a Windows VM  — and was pretty fun to use and good-looking.  I ditched Toodledo because the implementations were different on Android and other platforms, but mainly because Toodledoo didn’t treat Goals, Projects, Tasks, and Subtasks as if they were all fundamentally the same thing.

Any Lisp programmer will know what I mean (and I was one once): they’re all nodes and lists.

Todoist wasn’t perfect in this respect.  It had Projects, which could be organized into hiearchies, and it had Tasks which could also be organized into hierarchies.  But the two were quite distinct from the user’s point of view, and I can only imagine how different they were in the implementation.

But it was nice to have hierachies: it meant I could break a multi-step thing down into subsidiary steps for at least a few layers, and it meant that my Projects could contribute to goals, which were just Projects way up in the hierarchy.

Then My Life Organized (MLO) entered my life, and I’m not looking back.

At first blush, MLO is a step backward.  It’s essentially a desktop app with app versions for mobile devices (and, sadly, no version for the Mac… yet?).

I’m not a fanatic about the cloud.  My main utils for the cloud have to do with talking to phones and tablets, which seem to have a hard time syncing local copies of the data and work well with the cloud.

I do like having the same functionality on every device, and the lack of an MLO Mac version may do me in eventually.

But what makes me delighted with MLO is that the whole system is a hiearchy of tasks, from topmost goals to “next actions”.  I love it.

Why is this great?  You can go straight from a top-level “task” of “be sparkling” to “write great stuff” to “write the Great American Novel” to “Vomit out the first draft” to “Finish Chapter 1” to “Research Venusian bathing practices”.  And as you check stuff off, you pop back up.

Moreover, MLO allows you to rate the importance of a task (and its urgency, a nice distinction from 7 Habits days) relative to its parent, which allows you to get a very nice linear view of all your “next actions” sorted by filtered importance/urgency in the hierarchy.  A task that is desperately important to a goal that is “meh” will fall lower than an “average” task relative to a more important goal.

I gotta be honest.  A lot of this stuff seems cribbed from “Life Balance”, an app I used with great pleasure maybe 10 years ago but which didn’t make the transition to the multi-device/sync/cloud world very well.  Unlike Dropbox, Life Balance sync didn’t “just happen”, it took a lot of work, and a lot of failed attempts to sync.

But Life Balance was the first time I ran across the top-to-bottom hierarchy and the relative importance slider, and it won my heart.

(For all I know, LlamaGraphics got the idea from somewhere else.)

If MLO got the idea from Life Balance, they just ‘fess up to it, be generous, and move on.  They’ve taken the idea a long way.

More on MLO in coming posts, I’m sure, but for now, I’m dewy-eyed in love.