Music library management

bliss is almost ready for release. Our intrepid beta testers and I have been throwing it all at bliss over the past few weeks. We've found some bugs and also the time to add some extra features.

I thought it was a good time to show you how bliss manages album art for my music collection

I want my music collection to contain album art. I want it all embedded in the music files, and I want it all saved in a file prefixed 'cover' (this is a Squeezebox thing). I don't care about the size of the art, but I'd rather have something.

All these requirements are defined in bliss's cover art 'rule'. Here's how it looks:
Now I click 'Apply', and away goes bliss! bliss finds existing art in the library and makes sure it is embedded. Sometimes there is no art, but it finds it automatically (by downloading it from the Web) and then embeds it...


Some it isn't sure about, because either the tag doesn't describe the album properly or it's a rare recording, so bliss asks me...

And it's done! That took a fraction of the time it used to take. No searching through Google or Amazon. No using a tag editor to embed the image I found in each file.

The best bit? If I change how I want my album art stored I just change the rule and let bliss do all the work!

Reminds me of a song...

When building a digital music library, it is important to store music files in a consistent and useful manner.

Digital music is stored in files on your computer's hard disk, just like the documents you may create in Microsoft Word or the photos you take with your digital camera. These files contain both the music and the 'tags' that describe the music. When digital music is purchased via download, or purchased via CD and ripped, it is saved in such a file in a predetermined directory structure. For instance, when purchasing from the Amazon MP3 store the files stored in a directory for the album, which is stored in a directory for the music artist. For example, look how Ludovico Einaudi's beautiful Una Mattina record is structured when purchased from Amazon:


The tracks that make up the record are stored in a directory named after the record 'Una Mattina' which in turn is stored in a directory named after the composer 'Ludovico Einaudi'. This is how Amazon organises its downloads, but other configurations are possible when, for instance, you rip your own CDs.

At first, it's best to fire up your music player of choice and allow it to work out how your music is organised. However, at some point managing a large music collection leads to working with music at a file level. Sometimes it's a music player which cannot correctly interpret tags. Sometimes you just don't like the way the files are organised.

Rule based management is the best way of automatically organising a music library.

The same rules that govern tags, saved within music files, should also be adopted in structuring the very same files on your computer's hard disk. This leads to a consistent reflection of your music library within both your music players and also your file system. The rules should be expressed in general terms, for example:

[artist]/[album]/[tracknumber]-[trackname]

Gives a similar structure as the Einaudi example above. The important thing is that the rule is automated so changing the format changes all of your music automatically. For instance, a flatter hierarchy could be achieved with the following rule:

[album]/[tracknumber]-[trackname]

Here, the artist is not recorded. In rule based music management changing the rule is all you need to do... the music is moved for you!

File structure rules give more benefits:
* Control over disallowed or undesirable characters in files (maybe you don't like spaces in filenames, for instance)
* How the structure should work in ambiguous situations (what artist directory to use for compilation records?)

Importantly, the file structure is still driven by the tags and the automatic rules governing the music library in general.

Managing music files is an important task in music library management and with large collections it is best done with automated rules.

I listen as well as code, y'know. I've actually got a theory that programmers are generally music lovers. There's plenty of evidence of the connection between music and mathematics, and many of my previous colleagues have been avid music lovers with tastes spreading across many genres.

Anyway, what's been playing on the Squeezebox the past month or two?

I've enjoyed Metronomy's Nights Out. It's intelligent, adult pop music with some fantastic, catchy melodies. Particularly catchy are "My Heart Rate Rapid" and "On The Motorway".

After the Mercury Music Prize nominations were made I took a listen to a few nominees. I like Florence and the Machine's Lungs and The Horrors' Primary Colours. The latter is like My Bloody Valentine with a pop vocal laid over the top, a little like The Jesus and Mary Chain.

A bit old this one, but I've just caught up with MGMT's Oracular Spectacular. I love the laid back hedonistic message of Time to Pretend, sung through a pursed accent. Whenever I hear it I think of the cult movie The Warriors. I think the music shares a certain gritty yet stylised New York aesthetic.

Finally, on the classical front I've been listening to Ludovico Einaudi's Una Mattina. This is a minimal piano album, delicately phrased and perfect for background contemplation music... although to be fair it deserves a lot more concentration than that!

We have just uploaded a new version of the bliss website containing more information about bliss and, importantly, how to get involved in the beta programme - http://www.blisshq.com

The beta simply downloads cover art from the Internet. When you start bliss, and after you point it to your music library, it will look for all the cover art missing in your collection and attempt to download it.

I've already posted that bliss works differently to most music organisers. As well as being rule based (not letting you interact directly with your music, yet) bliss can also be placed wherever you like and accessed via its Web based user interface. It is also designed to be run and left alone; it is capable of noticing changes to your music library and firing the rules you have configured as a result. So, for instance, if you store your music on a separate server, you could install bliss there, leave it running and access it remotely. You could upload new music to the server safe in the knowledge that bliss will notice the new music and make sure your rules are adhered to.

We are looking for two main things from our beta testers:

1. Feedback as to how well bliss is monitoring the music library and downloading cover art
   
2. Ideas as to how to improve bliss and take it to the point that it is a releasable product

In return for your time we are offering full licenses to use bliss and, once the product is released, a one year subscription to get all the new features released in that time.

You may ask why we are releasing a rather cut down version of bliss. It's a fair question. I've already blogged about some of the many, many problems that exist in music library management, so why not wait until we've solved all of them? The reason is that the simpler that bliss is initially, the quicker we'll have something validated and therefore a base on which to build, and build quickly. We also need feedback from you, music lovers and music library owners, as to the most important problems to solve first. After the beta programme, and once we have released bliss fully, we will be releasing updates once per month with new features.

Would you like to beta test bliss? If so, pop along to the signup page and enter your email address. We'll get back to you shortly with how to begin.

I've made reference to 'rule based music library management' in previous blog posts. What does this mean? Fundamentally, rule based music library management changes the way digital music libraries are managed. Gone are the tedious, manual, error prone editing of music files. In its place, rule based management provides a fast, automated, consistent and correct approach.

The old way

Traditionally, digital music libraries are managed manually. The user decides how they want their music to be organised, works out what this means in terms of editing their music and then performs the editing.

For instance, you may decide they want to ensure all of your music fits inside eight defined genres. You must go through all of your music, ensuring each element of your library fits this goal. Later, if you want to add subgenres, or a new ninth genre, this means reorganising your entire library.

Popular software exists to manage music in this way, such as Mp3tag and Media Monkey.

How rule based management differs

Rule based management is very different. The user still formulates in their mind how they want their music library organised. However, they don't then go and apply this themselves. Instead, they describe their desired music library in the form of rules which govern the music on their behalf.

This reduces an enormous amount of the manual work involved in music library management. Indeed, large libraries require so much management it is the only approach that is feasible.

The obvious initial benefit is reduced time managing a library. Changing the way the music is organised is as simple as changing the rule. The rule then changes all the music on the user's behalf.

Further, rule based management eliminates inconsistencies and mistakes likely to creep in when manually organising a music library. To keep the library consistent the user must rigorously apply their desired policy to each music file in the library. Unfortunately, this is not one of the tasks humans are best suited for. We get bored and make mistakes (I know I do!). Software, on the other hand, is excellent at this, and thus software based rules remove the likelihood of making mistakes and introducing inconsistencies.

Practically, both approaches may be used. It makes little sense to ignore the excellent software already produced in this area. However, as music libraries become ever larger, our approach to managing them must change.

Rule based library management has a big part to play in the future of digital music library management.

I've just discovered Digital Music Collector. The author is working on an interesting project called MusicGenreTree, a community project to construct a complete tree of musical genres with more information besides. It'll be really interesting to see how they get on.

Maybe bliss could categorise music according to this tree? In the post Defining a standard music genre tree the author describes three existing genre trees available on the Web (and I added a fourth, allmusic, in Genre: separating your 'epic doom metal' from your 'free funk jazz'). It would be great to be able to pick the tree of your choice, and have all your music categorised against it automatically.

Here's a sample of the genres currently reported in my music library:

Alt. Rock Alternative Alternative Pop Alternative & Punk Alternative Rap Ambient Ambient Pop, Space Rock Art Rock Avantgarde Ballet Blues Brasileira Britpop Childrens

Dance Dance & DJ Dance Hall Disco Drum & Bass Easy Listening Electro Electronic Electronica & Dance Electropop Folk Funk General General Jazz

Hopefully it's clear that genres here are, well, a mess. If I want to browse my collection of electronica I not only need to view the albums in the 'Electronic' genre but also those in 'Electronica & Dance', 'Electropop', 'Electro' and more... and that's ignoring 'General'!

Poorly categorised genres make it more difficult to select the music you want to play.

There are many reasons why the list above makes life difficult.

The 'I give up' and 'just plain wrong' problems

These are albums and tracks that are just incorrectly tagged, or tagged so generally as to make the tag meaningless. This can occur when the tags are looked up from a public database with incorrect data.

It is common for community created music databases to have incorrect genre information, and these mistakes end up in your music library.

There isn't much of an excuse for this; the genre, simply, should at least vaguely describe the style of music.

The 'hip hop' or 'acid-dubstep-revival' problems

Genres often appear at wildly varying levels of detail. Most music genres have sub genres, sub-sub genres and... you get the picture.

Now this isn't 'wrong' per se, but it does make your music collection more difficult to browse and it does look plain ugly.

Genres populated from community databases have inconsistent levels of detail.

Typically I find myself wanting different levels of granularity at different times; it entirely depends on what I want to listen to. Thus, while having very general genres such as 'rock', 'pop', 'classical' and so on may look tidier, they don't provide the detail you sometimes need.

A further improvement rarely found in automatically tagged music is to specify multiple genres. The approach to this depends on your music player of choice and the underlying file formats being used to store the music. When a piece of music is tagged both with a general and a specific genre it means you can both browse for general and specific styles.

Using rules to gain control over genres

To gain a correct and consistent view of your music collection it is important to use both an authoritative source and to specify a consistent level of detail to be applied across your collection.

Authoritative, reviewed sources are available such as allmusic and MusicBrainz, both of which have more accurate data than the other commonly used databases such as CDDB and FreeDB. Furthermore, allmusic offers a comprehensive genre tree which, ideally, one could adopt to categorise their library.

Once you have decided how you want genres to appear, its time to put this into action. This is where you realise you have 10,000 tracks to tag, and you better get each one right. Best put the coffee on, it's going to be a long night!

Using authoritative sources for music information should be enforced via rules over your entire library, because editing the music directly is both tiresome and prone to mistakes.