07 January 2009

Liferea Development Failures So Far

From time to time it helps to look back and to reflect on the results so far. When doing so you do sum up your success stories and your significant/total failures. I want to focus on the latter ones for this post.

Liferea v1.2

Let's exclude the v1.0 branch as the initial implementation of features and start with the goals and their results of the v1.2 line:

  • Fixing 64bit Instabilites: In fact v1.0 was unusable on x86_64. During the year to reach v1.2 a lot of patches and debugging went into the stability issues and reduced the crashes. But never completely, v1.2 still crashed. Later when the last issue was fixed during v1.4 preparation the most crashes where caused by the HTTP-client code.

    What should have been done: not having own HTTP-client code would have skipped almost all of the 64bit problems. But due to missing GPL'd network libraries for C supporting cookies this was not possible at the time. Right now v1.5 switched to libcurl.

  • Correct Locking: With v1.0 a lot of users had problems with the locking of the cache directory (XML files that only one instance could write at a time). Crashes often lead to stale lock files and the running instance detection not working perfectly often prevented removing the lock file. The result was an annoying dialog warning of a (not really) running instance.

    What should have been done: Use a standard locking mechanism ripped from another reliable open source project. Done after 2 years later by using BaconMessageConnection from libbacon of the GNOME project with v1.4.

Liferea v1.4

  • XulRunner Flash Crashes: Starting with Flash 9 (now 10) Liferea when running Flash applets in the GtkMozEmbed widget on 64bit systems very often crashed and as often just froze the application. Both behaviours unacceptable and causing dozens of bug reports.

    What should have been done: Note: there is no reasonable way to disable Flash in GtkMozEmbed. So the only thing left is a HTML postprocessing stripping all <embed> and <object> tags to not even allow execution. Sadly it's still not realized. Also starting with v1.5 the renderer is switched to Webkit which is much more stable and faster than GtkMozEmbed.

  • Fixing Slow XML Cache: That's the #1 failure in the Liferea development so far. After a lot of discussion in the mailing list a lot of people hinted about using sqlite as a leight-weight DB backend. And in fact coding against sqlite is pretty simple, runtime stability and version compatibility is good too. The thing is you have to know about the downsides of sqlite:

    • Heavy disk activity (Firefox3 Places users are suffering too).
    • Continuing DB fragmentation and no runtime auto-reorganization.
    • Very long startup time for fragmented DBs (e.g. 30s for 50MB).
    • VACUUM (the manual reorganization SQL command) not working when you JOIN on ROWIDs. This I never read in the sqlite documentation but a user gave this hint.

    Right now there are many users dropping Liferea for excessive startup times and temporary freezing when doing massive feed updates.

    What should have been done: I'm not even sure. The XML based cache was as slow as sqlite is now when it got large enough and it also caused a lot of disk accesses. The only really advantage of the DB is easier search folder rule implementation and simple transaction based migration.
Liferea 1.8 or 2.0?

So what's in for the next release cycle? When asking what users would care most about it then it is PERFORMANCE. The question remains what to change? Right now I think using sqlite is not wrong itself, but the schema might have to be changed significantly and the expensive use cases (search folders rules) should not be realized using SQL views. To change this the feature and others might need to be removed altogether. Additionally in-memory caching of items (e.g. all unread) might be wise as they will be accessed with a high-propability anyway. When thinking about it changing the features in favour of performance means having a v2.0 line mercilessly reducing features eliminating everything effecting runtime performance.

Feel invited to comment your opinions and experiences!

11 comments:

Anonymous said...

Are you going to remove Flash support altogether? I really like using Liferea, including its flash support! In fact it's the reason I switched from Akregator.

Mike

Anonymous said...

There seem to be many people unhappy with current Liferea performance, some attempts to write a new reader.

Why not call for a "Liferea 2.0"-team, and see, who would like to get involved?

(This should include a major rewrite of course, so everyone is able to step in, and more people will work on it that way).

djcb said...

regardless of all that, I'd just like to say that Liferea is a very nice feedreader -- thanks!

regarding the peformance problems, are there any traces (oprofile, sysprof) to pinpoint the hotspots?

Anthony said...

I have to say, all the features I could possibly need are there, and the interface is great - performance should definitely be the top priority for the next release. Keep up the good work!

Lars said...

Thanks to all of you for your feedback!

To answer the questions:

Mike: Flash stripping would be a preference option that is enabled by default. Brave users could turn it off

anonymous: Good idea. I'll write up another posts asking for people to participate in a complete restart.

djcb: The exact performance bottlenecks are pretty well known. I have profiled using oprofile several time. The thing is to solve them you need a different design :-)

Mike said...

Lars: Thanks, that freedom of choice would be great. My experience with Flash in Liferea is as follows: In Ubuntu prior to inteprid I had many crashes related to Flash (same for Firefox, btw), but now on Ubuntu intrepid I'm very happy with it and I watch some video podcasts regularly within Liferea and no crashed. It seems the crashes were related to the sound configuration of Flash (PulseAudio woes.)

The Liferea package from intrepid (1.4.18-0ubuntu2) worked very well, but I recently switched to a backport from Ubuntu jaunty (1.4.23) because I didn't like the GUI for changed items in 1.4.18. Still no Flash problems anymore :)

Mike

iElectric said...

maybe CouchDB would have been the right choice for XML storage?

I'm satisfied with startup performance though.

Anonymous said...

Hi,

@ the second comment:
Anonymous said...
Why not call for a "Liferea 2.0"-team, and see, who would like to get involved?

Given that no-one has been interested in joining the development up until now, I dont think it will happen just because there is a re-announcement.


I also find the performance being the most important factor, right now, Liferea is very sluggish. Often it stalls for several seconds.

Additional work on the google integration is also something I look forward to.

Good luck and thank you for your perseverance.

Heni

Anonymous said...

I would like to see liferea-add-feed work without liferea already being loaded (it should load liferea if necessary).

Lars said...

Anonymous: No bug reports in the blog! Please add one at SF. Please also provide the console messages when running "liferea-add-feed" because in principle it does start Liferea.

Anonymous said...

i used to use liferea from 2007 until december 2008 then i switched to google reader.

what all i need is (1) i still can reach my important rss (starred on google reader)
while i have my pc or not and (2) i still could reach my older rss when i need..

i know my 1st requirement is being delivered, but i had no idea with the 2nd requirement

the decision of switch is made when i realized i loss my older item when i searched post that i know i ever read it before... (the most bottom post of that subscription is only 2 month before, while i had been subscribe for almost 2 years)

i still put my hope on liferea...

PS: is there any chance using embedded mysql like amarok to subtitute xml/sqlite?