Music library management

Storing music in FLAC format is an excellent choice. It's higher quality than MP3 and, importantly, is 'lossless' which means you can always convert it to another format. Not all players support it though, so sometimes you need to have your same music in a different format. How do you accomplish this while not wasting time or storage space? mp3fs is a fully automatic and storage efficient way of keeping a copy of your FLAC library in MP3 format.

What are the other options for converting from FLAC to MP3? First, it can be done manually on a file by file or bulk basis. Some converters (or 'transcoders') work with a GUI while some work on the command line. Examples of both are dBpoweramp Music Converter and flac2mp3.

There are two problems with the manual approaches:

1. They have to be run each time your music library changes, because they are manual. This can be mitigated by running the job periodically
   
2. They use more storage space, because a separate copy of your FLACs in MP3 format are kept

Instead, on my Linux home server, I use mp3fs. mp3fs represents your FLAC folders as MP3 folders. You simply point mp3fs at your FLAC directory and tell it where the resulting MP3 files should end up. No MP3 files are created as such, it just looks like they are.

First decide where you want your mp3s to be stored. Let's say you want them stored in /mnt/music/mp3. This should be an empty directory. Now, choose your FLAC directory. Let's say this is /mnt/music/flac:

gravelld@gravelld-laptop:~$ ls /mnt/music/flac
Alison_Balsom    Mint Royale    Oasis     Suede                  Various
Louis_Armstrong  Nelly_Furtado  Pendulum  The_Chemical_Brothers

Meanwhile:

gravelld@gravelld-laptop:~$ ls /mnt/music/mp3
gravelld@gravelld-laptop:~$

Nothing in there yet! So, let's weave some mp3fs magic:

sudo mp3fs /mnt/music/flac,192 /mnt/music/mp3

Now:

gravelld@gravelld-laptop:~$ ls /mnt/music/mp3
Alison_Balsom    Mint Royale    Oasis     Suede                  Various
Louis_Armstrong  Nelly_Furtado  Pendulum  The_Chemical_Brothers

And interestingly:

gravelld@gravelld-laptop:~$ sudo ls /mnt/music/mp3/Oasis/Definitely\ Maybe
01-Rock 'N' Roll Star.mp3  07-Bring It On Down.mp3
02-Shakermaker.mp3         08-Cigarettes & Alcohol.mp3
03-Live Forever.mp3        09-Digsy's Dinner.mp3
04-Up In The Sky.mp3       10-Slide Away.mp3
05-Columbia.mp3            11-Married With Children.mp3
06-Supersonic.mp3          cover.jpg

These are FLAC files stored in the FLAC directory, but now they appear to be MP3 files too. The important bit here is that these MP3 files are not actually stored on the hard disk. Rather, each time you access the MP3 directory or files therein, mp3fs mimics the directory structure or creates the MP3s on-the-fly. This means:

1. You don't need to tell mp3fs when the music library has changed
2. There's no extra storage space required to store your MP3 mirror of your FLAC files

Some people say "storage is cheap". I agree, but your time isn't. Storing two separate trees of music is duplication and therefore imposes a management cost. It's not just adding music either, it's deleting it and modifying it too that would, with a manual solution, require a rebuild of your MP3s.

That is how I mirror my FLACs to MP3s on my Linux home server, and I haven't had to think about it again since.

I'm a process automation obsessive. Last week I started a three part series on improving my digital music workflow. I wanted to automate more parts of my workflow, from this:
To this:
Five steps down to one!

Recognising that VortexBox will do a large part of this for me, I installed VortexBox in my home network within a VMware 'virtual machine' and got VortexBox up and running. This is all pointless without having any music to play of course, so this week I integrate VortexBox into my existing music network by linking my digital music files to VortexBox. I then describe where bliss fits into the workflow, and finally and most importantly I get to play some music!

A quick warning: this post contains some specific Linux related techie language! All this is possible within Windows and Mac environments too, but my chosen home network technologies tend to be Linux based ones.

We already know from my previous post that VortexBox contains a Squeezebox Server for distributing music around the home. A Squeezebox Server is a piece of software that plays music from a directory on your computer's hard disc. So, to integrate my newly installed VortexBox with internal Squeezebox Server with my existing music, I need to link the nominated directory with my music. My music is already available from my NAS via NFS, so all I need do is 'mount' the NAS's music directory to the nominated directory on the VortexBox. To do this, first I make the music available from the NAS, then I perform the mount on the VortexBox to access the music.

To make the music available from the NAS I add an entry to /etc/exports, which lists all directories made available via NFS:

/mnt/media/music 192.168.0.5(rw,async,no_subtree_check)

This means: make the '/mnt/media/music' directory available to the computer with IP address 192.168.0.5 (the VortexBox, since I've configured it to have a static IP). The music files are available for reading and writing. More on the /etc/exports configuration file here. Then, I reload the configuration file as so:

sudo /etc/init.d/nfs-kernel-server reload

The music is now available from the NAS. Now, I need to mount this music. The directory that the VortexBox assigns to play music from is, by default, /storage/music/flac. So, to mount the music within the VortexBox:

mount 192.168.0.3:/mnt/media/music /storage/music/flac

This means: mount the '/mnt/media/music' filesystem on 192.168.0.3 (the NAS) at /storage/music/flac. Listing the contents of /storage/music/flac lists our music successfully! Now, I don't want to re-issue that mount command every time the VortexBox starts, so I add a line to /etc/fstab:

192.168.0.3:/mnt/media/music /storage/music/flac nfs rw,noatime 0 0

Now, every time VortexBox starts, it will mount the /storage/music/flac directory automatically so my music plays with no effort.

With the directory mounted, and the music files visible on the VortexBox, we can now tell the VortexBox's Squeezebox Server to 'rescan'. Rescanning is the process the Squeezebox Server undertakes to update its internal database and make music available for playback.



Where does bliss fit into this? bliss isn't dependent on Squeezebox Server or VortexBox, it works on the digital music files directly. As a result, I have kept bliss running on the NAS, co-located with the music files, rather than install it in VortexBox for now. That said, either configuration would be acceptable. Whenever I move music onto the NAS, bliss recognises the music and conforms the album art against my rules automatically. For instance, if I move an album over with no embedded album art but with a co-located image file, the image file gets embedded into the music files without me doing anything.

Now... the important bit - listening to the music! I started up SqueezePlay to test my work. After choosing the new Squeezebox Server with the 'Choose source' function I soon had music streaming in full FLAC quality.


And there we have VortexBox streaming my music around my house once more. Next week we complete the final steps in our digital music workflow when we setup ripping from CD, automated tagging and album art, and automatic rescanning of the Squeezebox Server.

In the beginning (well, 2002 anyway), there was the computer. The computer was very useful. You could read and write documents, listen to music, play videos and reminisce with embarrassing old photographs.

Over time, all this data began to grow in size. It became difficult to fit it onto a hard disk on a standard computer. We began to want to backup the data, because losing precious music and photos would be A Bad Thing. Then, as we added more computers to our home in the shape of netbooks, smartphones and tablet computers, we wanted access to all this data from all these devices.

To solve this, we decided to centralise the data.

We moved all our data to computers responsible simply for data storage. These computers are called NASes (Network Attached Storage). They offered far higher capacities, could provide your documents, music, photos and video to whatever computer in your house requested it, and were easier to backup. They tended to run continuously at a low power draw, so as soon as you wanted your data, there it was. Now, companies like Buffalo, Linksys and LaCie make affordable, easy to use home NASes that provide all of these benefits.

This worked well until we found it wasn't just the data that we wanted to centralise in our homes, but the actual applications too. For instance, music streaming servers like Squeezebox and Personal Video Recorders (PVRs) like MythTV would work, in the background, while you got on with your life. This new wave appears just ready to break through. Microsoft's Windows Home Server product is a version of Windows that you can install on servers. Linux is another popular option amongst hobbyists, as is the Mac Mini.

Where does music management fit into this?

There's no reason why music management should be something a collector has to laboriously step through on a constant basis. If we define rules by which our music should be managed, music management can be done on a continual, fully automatic basis. When new music is added to the collection, rules kick-in to ensure that the music library is maintained in a consistent, complete and correct state. If the application of the rules is not certain, then we can ask a human via normal methods such as email or a Web interface, as humans are great at creativity and problem solving.

So, the result is much less time tagging and more reliable and consistent results.

bliss is written to run on a home server.

This is why it has a web interface, so that it can be reached from anywhere in your home network. It is also why, most importantly, it applies the rules you define when new music is added to your collection. This way, you dictate the rules, and bliss does all the hard work.

Where will home data storage go next? Some say 'the cloud', a vague, loaded term that tends to refer to online storage of data over the Internet (and lots more besides). While this is an excellent solution for a number of reasons, as yet such an infrastructure is not capable of storing data in all its guises, for instance the lossless music beloved of audiophiles is simply too large both in terms of the capacity required to hold it and the bandwidth required to upload/download.

It'll be interesting to see in what directions home servers go, but so far it appears it will make music library management quicker and more accurate.