Future updates to the import routine

It's funny, but one of the most-used features I never intended to add to Firefly III is the import routine. I (still) believe that you should add all of your transactions by hand. Yes, even when you want to go back 2, 3 or 5 years. This is something I wrote about extensively and I won't bore you with it. 

However, since so many people asked (and I like a challenge) I first built and released the CSV import routine some time around 2015.

Challenges

The import routine is also my biggest current challenge in developing and maintaining Firefly III. There's always something changing in the bunq API. The rules for FinTS compatible banks have changed in light of the PSD2, complicating the import as well. And of course, I've yet to see a bank that can generate a consistent and logical CSV file.

These challenges often forced me to release a new version of Firefly III, just to fix the import routine. This is terrible for the user experience, bad for my flow and annoying in general.

API

Another thing I never expected to add to Firefly III was the API. Often requested I was hesitant initially, leaving user data exposed through scary API's. But I got better and nowadays Firefly III features an API that isn't half-bad. 

Future updates

To keep the import routine(s) future proof I'm going to leverage the API, and build separate import applications that are compatible with Firefly III.

Instead of having the import routines be a part of Firefly III, these will (over time) become their own separate applications that leverage the Firefly III API to store and validate new data. Using these new applications you can import data from CSV, bunq or FinTS, just as you currently do.

Changes

This is quite a big change, that will play out over the next few months. And of course, I won't disable anything in Firefly III before its replacement is completely ready. Never throw out your old shoes before you have new ones.

The advantages to this approach are many:

  • Docker users are no longer bound to the web interface (which tends to timeout)
  • You can import remotely, hardening your Firefly III installation.
  • Separate release tracks mean that issues will be fixed quicker and more easily.
  • Small and smart applications are easy to run and fast to deploy.
  • A working import routine for bank A, can serve as an example for bank B.
  • It's easier for other users to write their own import routines.

Like I said, this is something that will play out over the next few months. On GitHub, you can already find the skeleton code for the new CSV importer, which I invite you to check out. I can guarantee you it won't work but if I piqued your curiosity, you're as thrilled as I am to move in this direction.

Keep in mind however, that this means I may not fix imminent bugs and issues in the current, Firefly III-bound import routines. I would never leave you hanging of course, but new features will have to wait for the new import applications.

I can't say often enough how happy I am with a warm, welcoming and active community like yours, and I for one look forward to the coming months.

Cheers,
James

Tier Benefits
Recent Posts