Journaling

A little over ten years ago I decided to try using a journal app to record daily events, and I selected the Day One app, as it allowed me to enter events on my Mac, my iPhone, or my iPad, and then sync them across the devices. It did this flawlessly. In truth I did most of my writing on the Mac, as that’s where I felt most comfortable typing, but if I went out to eat, I could start an entry at the restaurant on the phone, including what I ordered, and then when I got home I finished the entry on the Mac by editing it into a more flowing bit of text.

This worked fine for several years until Day One pulled the rug out from under me by going to a subscription model. In other words, I’d have to pay a yearly fee to keep using it. I hate subscription models, so I went in search of another app, but I couldn’t find one that worked to my satisfaction.

I might very well have succumbed to Day One’s subscription plan except for one thing: just as the old app was expiring, meaning the I’d have no choice but to move to the new app, all the data that I had entered into Day One disappeared! All several years worth. Their database file that was kept in the Mac’s normally hidden Library folder simply vanished without a trace. For some reason none of my backup apps had kept a backup of it either. The syncing to iPhone and iPad had already stopped, so I couldn’t get the data from them.

I would have been in total despair except that a few months earlier, because I had been searching for a replacement app, I had exported all the data, so I still had the exported file. Thus, I had only really lost a few months of journaling data.

Reluctantly I went with the free version of the Day One app. It allowed one to make entries only on the Mac and it didn’t have most of the bells and whistles that the paid version had, but I thought it was better than nothing until I found something better. As I didn’t trust it to keep my data, I exported my data regularly.

Then Apple came out with its own Journal app a couple years ago. Alas, it was only available on the iPhone, so it wasn’t of much use to me. But recently I discovered that it is now available on all its platforms, so I wanted to give it a try if there was a way to import all my Day One data.

Happily a web search revealed that Felix Wong had developed a method for doing exactly that!

Felix Wong's blog.

His method required one to 1) export the data from Day One into the JSON format, 2) parse the JSON with AI, and 3) import into Apple’s Journal app with an Apple Shortcut that he had written and which he made available on his site for download.

There were a couple problems with his method from my point of view. First, I don’t do AI. Period. And second, he only had about 500 entries to export and import, whereas I had nearly 5000.

The first hurdle was relatively easily solved. Instead of parsing the exported data with AI, I merely had to write a simple shell script to parse the data. In doing so, I discovered that it was easier to parse a Plain Text exported file than a JSON file, and a little bit of testing revealed that Apple Journal was quite happy with the date format that Day One used for its Plain Text export, so that simplified the shell script even further. The main problem was that I was a bit rusty in writing shell scripts, so what should have been a half-hour’s work turned into—well, never mind.

Once I had parsed the export into the individual files that Felix’s Shortcut expected, I tried running the Shortcut. Except I didn’t try to run it on all 4925 entries at one time. Felix ran it on his iPad and it took an hour to do his 500 entries; I’d be running it on my Mac Studio so it should go much faster but still. I tried it on the first 500 and that took about ten minutes. So far so good.

But it got seriously bogged down after that. I mean seriously. When it got to about 750 entries or so it really started to choke, and I couldn’t do anything else while the Shortcut was running on the Mac. Each additional entry was taking over 30 seconds to be added.

Felix’s Shortcut put a Title on each entry. As I had no use for a Title, in my shell script I simply put a space character where a Title should be. What if Apple’s Journal was choking on all those spaces? If it was indexing on the Title and all the Titles were exactly the same, perhaps the indexing was what was bogging things down. So I rewrote the shell script to make the Title the same as the filename. Each file name was simply a sequential number starting with one, so that’s what the Titles were. This had the pleasant side effect that I could see exactly which file was being added as the script ran. 

Alas, it didn’t solve the problem.

So once I got into the 700s, I tried running the Shortcut with only ten files at a time. It took about ten minutes to add ten files this way, and I couldn’t use the Mac for anything else while the script was running.

What if I turned off iCloud for the Journal app? Maybe sending each entry to the cloud was what was slowing it down. Nope. That didn’t help. Nor could I simply walk away from the Mac and let the Shortcut run, for example overnight. When the screensaver kicked in the Shortcut stopped running. OK, turn off the screensaver and turn off the display. No that wouldn’t work either because sometimes the Shortcut seemed to stop on its own, as if it ran out of breath and needed a kick to start up again.

I finally realized that I had a second Mac. An old iMac Pro with the M1 chip. I don’t use it for very much anymore, but at least I could run the Shortcut on it and not have it bog down my main system. So I copied everything over to the iMac Pro.

I expected the Shortcut to run even slower there than it did on my relatively new Mac Studio, but wonder of wonders, it ran like a champ!

And then it dawned on me.

During all this time I had never quit the Journal app!

Apparently that’s all it took. Run the Shortcut on a bunch of files, say a hundred at a time, then quit the Journal app, repeat. 

I got all 4925 entries imported into the Apple Journal app, and thanks to iCloud they’re available on all my Apple platforms.

Oh, yeah, I had eight photos among all those entries. Like Felix I simply added them into Apple Journal manually. Happily, when they exported, they carried their date stamps with them, so finding the correct entry was fairly simple.

BTW, all those entries comprise over 200,000 words all told. You probably didn’t realize that I can be a bit verbose at times.

And I owe it all to Felix Wong’s Shortcut.

Thank you, Felix!

Now I have to get back into the habit of writing Journal entries regularly and remembering that I can use my iPhone and iPad.

Leave a Reply