• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

"Pick the right library for your problem"

 
Sheriff
Posts: 7001
6
Eclipse IDE Python C++ Debian Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
While browsing the web page for the book "Software Mistakes and Tradeoffs" I noticed that one of the bullet points indicated that a reader will learn how to "Pick the right library for your problem".

This fascinates me. So much so that I am currently doing a PhD on it!

I would love to read and discuss how people deal with the huge problem of selecting the right library, particularly when faced with thousands of options to choose from, hardly any of which provide any useful information to help with the process.

As an example of the scale of the problem, do a GitHub search for "template engine". It currently returns well over 10,000 matches. I have set up a test rig to evaluate a (tiny by comparison) selection of these, and found that there is over a 1000 x performance difference even across my small sample.

Every single one of the template engines tested claimed to be fast and powerful

How are we ever supposed to get past this problem?

(BTW See my sig for more details of my research)
 
Saloon Keeper
Posts: 27865
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Funny that you should ask that. Disclaimer, though. "Right" means about as much as "Best" here, and to me "Best" is a 4-letter word.

But is this a "library", or is this a platform? Your choices for libraries are rarely that rich once you've got a language environment fixed.

I maintain a private GitHub-style repository on my server farm that I use for personal infrastructure change tracking and archiving and as a communications medium for client projects. For years I've been using Gogs. It's available in container form, simple to setup and open-source, as I prefer my software to be.

But it has one fatal flaw. When started cold, it does a scan-and-repair of all the archives and there are now thousands of files in those archives when you count the git meta-information. And that can take FOREVER.

I had to rebuild the server that the Gogs container runs on the other day and after 14 hours, I started looking for an alternative. Previously, I'd patched a script to avoid redundant fix operations, but the latest release of Gogs no longer has that script and I didn't want to have to make mods to a project written in Go.

Gogs is a great product, but unfortunately, lacks timely support. As I understand it, the author is not fluent in English (Japanese) and only works on it when convenient.

So I fired up Google and looked for possible replacements. After throwing out the obvious disqualifiers, I ended up with 2 and at that point it was almost a coin flip. One was in Java, the other in Go, but the one in Go (Gitea) is a fork of Gogs. Since Gogs has more fans than support, a bunch of people forked it and support it as a group. Now in theory, it also can migrate from Gogs, but that requires a Gogs backup and you can only backup Gogs from the webapp and the webapp only starts after all the file-scanning is done and the file-scanning showed no signs of completion after more than 24 hours, I've been having to pull projects from the Gogs repository files by main force and add them to Gitea manually. Could be worse, though. Gogs could have been using a non-standard repository file format.

So I hope that this resolves the problem. Otherwise, I do it all over again.

As a design note, it's NEVER a good idea for a webapp to spend much time in its initialization code. You should be able to do at least the bare minimum of web application after no more than a minute or so. This whole debacle could have been avoided if, instead of doing the file repair in the init code, it ran it as a background or just-in-time process.

Now, as to libraries, I did have a project that used Apache OpenJPA. I was happy with it until one day I needed a feature that hadn't yet been added. JPA, of course, is a standard, so I switched from OpenJPA to Hibernate and that was a job that could be done in less than half an hour. Why Hibernate? Mostly because "everybody uses Hibernate". Plus it had the needed feature. Otherwise, it would have been the next library I could find.

This works better with standard APIs than with generic stuff like, say building PDFs. In such a case, all you can do it look for essential features and discard all the candidates that don't make you feel comfortable in their support, maybe try a pilot project, and hope for the best.
 
Frank Carver
Sheriff
Posts: 7001
6
Eclipse IDE Python C++ Debian Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Picking the right components for a software system is always a problem, whatever level you look at it. To me the main issue is the lack of real information. Sure, there's plenty of "how tos", and tutorials, and code examples, and overexcited marketing material, but a distinct lack of concrete specifications and comparisons.

  • If you buy a car you want to know how many miles it does per gallon.
  • If you buy a kitchen appliance it usually comes with an energy rating
  • If you do any kind of electronics you always get a data sheet which goes into great detail

  • With software? Not so much.

    I completely agree that there is no such thing as "best", but in order to pick an appropriate component you need information. Without that, the choice is either to spend a lot of time (and money) evaluating lots of components, or select by "gut feel" or "word of mouth".

    Following on the "template engine" example. Even if we restrict the GitHub search to only include template engines in Java there are still several hundred to choose from. Admittedly, a lot of this is down to CodeRanch alum Lasse Koskela who wrote a book and used creating a template engine as a case study, but there are still a lot to choose from. I have spoken to many developers, and so far none have been able to come up with rational way to choose components.

    If you have ever worked with templated text in Java you have probably heard of Freemarker, Velocity, and Mustache. They get most of the coverage in articles and blog posts. They all do pretty much the same job.

    Any idea which is faster? Any idea which is most flexible?

    The same is true at an application level.

    Do you have any idea how much more electricity it takes to run a website using Wordpress compared to using a site generator and serving static files ?
    Do you know which is faster or which uses less energy when selecting between webservers such as Apache, Nginx, and Lighttpd ?

    Without this kind of information, how are engineers and software developers supposed to make sensible choices?
     
    Tim Holloway
    Saloon Keeper
    Posts: 27865
    196
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    What I did was google "X vs Y".

    What you generally get is reviews of general degrees of opinion/shilling and do a feature/performance compare against them to get a hopefully less-biased overview.

    In the case of performance, it's not a bad idea to search on "X benchmarks" and "X performance".

    It's important to note the dates of any reviews you read, though. Nothing worse than an outdated review.
     
    Ranch Hand
    Posts: 150
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    One thing I do is look for youtube videos demoing the functionality that I am looking for.

    It is not ideal but has helped me reject some libraries/tools that I would have downloaded and experimented with, just to find that they did not do the job.  Mainly I find that the video helps me filter out stuff that does not do what I want.  However, if the video shows that the software has/does what I need then I have a mini-tutorial to get me started!

    Downside - not every library has a video on youtube.
     
    Tim Holloway
    Saloon Keeper
    Posts: 27865
    196
    Android Eclipse IDE Tomcat Server Redhat Java Linux
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Fintan Conway wrote:
    Downside - not every library has a video on youtube.



    And that is just as well. YouTube is generally the LAST place I look. I haven't the patience and to me it's usually an indication that the presenter was too lazy to do a proper write-up. Subtract extra points if it's couched in a particular IDE or similar tool that I might not use.

    I generally find the Wikipedia to be a good place for executive summary information. A product has to be pretty obscure if it doesn't have a Wikipedia entry these days, even if it's just a posting of stuff from the marketing department (which generally gets tagged as such).
     
    With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
    reply
      Bookmark Topic Watch Topic
    • New Topic