Rebuilding My Music Library

Welcome to ODJT
Wanting to join the rest of our members? Sign up today!
Sign Up

DJ Bobcat

DJ Extraordinerror
Nov 8, 2014
6,389
9,534
115
Oklahoma City
#1
This is gonna be difficult. It’s also gonna take some time to describe where I am with the music library today and how I got here. I’ve been needing to rebuild, shrink, reorganize, and just generally clean out my music library for a LONG time. But it’s HARD. A LOT of work... takes a lot of time... and isn’t fun. I can usually find a more fun project to do, which keeps pushing this one to the back burner.

My current music library, a copy of which resides on each of my DJ computers, contains over 120,000 tracks. YES... They’re all legal! I started buying CD’s in the mid 80’s and collected thousands of them over the years (lost count at 8000)... LONG before I ever thought about being a DJ. In the late 90’s I built a home server, connected it to my stereo system and TV, and started digitizing my music. At first I just digitized and stored the tracks I liked, but then I found that it was a hassle to pull CD’s off the shelf when I wanted to listen to a whole album (CD), so I rescanned my entire collection in the early 2000’s. Originally, disk drives were small and expensive, but over time, capacities increased and prices declinesd, so multiple terabyte drive arrays became the norm. I’ve since gone through several home servers... mostly Linux, or some Unix based OS. The original copy of my music library still resides on my home server.

The music libraries on my DJ computers are all copies of the main library on my home server, only the folder structure differs slightly. In my main library the folder structure is alphabetical by ARTIST>>>ALBUM>>>TRACK TITLE. On my DJ computers, I have top level folders with each letter of the alphabet (A-Z) plus a folder (0) for artists with names starting with numbers (like 38 Special and 98 Degrees). Then the folders for each artist go into the folder with the appropriate letter of the alphabet matching the first letter of the artist’s name. I have all my songs tagged (MP3 tags) with vital information like Release Year, Genre, etc. but tracks are not sorted by genre or decade within the file structure.

To keep ALL the copies of the music library in sync, I have a STAGING folder on my main server. All my computers can access the Main Server. The Staging folder has sub folders labeled by the date the music in it was downloaded and/or ripped, then the folders within are the in the same Artist/Album/Track Title structure. Then I pull the files into each music library on each PC. Since all the computers are not connected to the network or booted up all the time, this process seems to work the best, but it IS time consuming.

So the issue is... too many tracks that will likely NEVER be played when DJing. My preferred style of music was generally mellower than the music I play at most paid DJ events, so probably half or more of the music needs to be deleted. Not a lot of call for John Gorka or Clannad.

How am I gonna do this??? I think the best way to start is by creating a new empty library on my Main Server. Then start copying files into it from the old library. But this is where it gets difficult. I guess I just do one artist at a time, trying to copy only the tracks that I might play when DJing... making sure I get all the tracks on my current playlists, and all the tracks on the lists I have of most requested songs. That could take FOREVER, but I can’t think of a better way.

Has anybody else gone through something like this? How did you do it? Is there a better file structure than what I’m using?

I’m looking for suggestions here. If this is gonna take FOREVER, I need to get started soon.


Sent from my iPhone using Tapatalk
 
Last edited:

ittigger

Hundred Acre Industry Icon
Feb 1, 2011
15,078
11,666
115
Western Maryland
#2
Like you, I should probably clean up my library. I think one of the reasons I have not cleaned it up thus far is (I'll use your example) .. if you're at an event where someone actually wants to hear John Gorka or Clannad (and you can play it) - then you don't have it with you. I'd rather have it and not need it - then need it and not have it.

I store things by Genre\Artist\Album\Track-Title-Artist.mp3
 

DJ Bobcat

DJ Extraordinerror
Nov 8, 2014
6,389
9,534
115
Oklahoma City
#3
Like you, I should probably clean up my library. I think one of the reasons I have not cleaned it up thus far is (I'll use your example) .. if you're at an event where someone actually wants to hear John Gorka or Clannad (and you can play it) - then you don't have it with you. I'd rather have it and not need it - then need it and not have it.

I store things by Genre\Artist\Album\Track-Title-Artist.mp3
I thought about sorting by genre, but I had albums with tracks of different genres, and I didn’t want to split the albums. I can sort by genre in my player software, so didn’t really seem necessary to have genre in the file structure.




Sent from my iPhone using Tapatalk
 
Likes: ittigger

ittigger

Hundred Acre Industry Icon
Feb 1, 2011
15,078
11,666
115
Western Maryland
#4
If the album is a 'Various Artists' album - which is usually where you would see different genre's, then that's the folder it goes in. :)

In that case, it would be Various Artists\Album\Track-Title-Artist.mp3

I can also sort by genre in my software - but way back when I started ripping, all this high tech stuff wasn't around and you couldn't sort by genre - and I needed to be able to find it through Windows as easily as looking through my crates. Along with that, I know how Windows acts when you put a ton of crap in one folder - so I was not going to store everything as Genre\Title-Artist.mp3 or even as Genre\Artist\Title.mp3
 
Last edited:
Likes: DJ Bobcat

steve149

Urbane Legend
Sep 26, 2011
20,851
29,626
115
Prospect, CT
#5
I use the iTunes default (Artist / Album) even for CDs I rip. Genres and such I do in iTunes as Playlists.

I am in the process of re-ripping my physical CDs to a lossless format (I've been using wav for now).
 

dbstudios

DJ I think
Aug 2, 2018
143
122
45
38
Walker Michigan
#6
I'd rather have it and not need it - then need it and not have it.
I have the same mentality, but I know my collection is severely lacking. I found out recently that I really need to organize my music by genre, and when I start adding more music, it will be easier to organize. Unfortunately I have just over 8k songs to sort through.
 

TwinSpinDJ

Up-lighting
Jan 21, 2010
2,513
1,645
115
Fayetteville, NC
www.twinspindj.com
#7
Just started ripping ALL of my Top Hits, Promo Only, and Prime Cuts into iTunes folder on my external HD in MP3 format. ITunes is great for doing lists and printing those lists...entire library, if necessary. I will then import select lists into my digital software program OTS. iTunes also does a wonderful job of searching for duplicates. Anyhow, the reason for ripping my professional DJ source CD is so that I can pack them up for storage...maybe even sell. Regardless, I have no need for them any longer now that I have the external HD. Ripping them at 320 CBR. Long project. It will be months before I'm done. Like 52 weeks x 20 years.
 

DJ Bobcat

DJ Extraordinerror
Nov 8, 2014
6,389
9,534
115
Oklahoma City
#8
I can see how having upper level genre folders could be useful, but it might be difficult to do, given the genre is in the MP3 tag. I probably know off the top of my head what the genre is for maybe half of the artist/albums, but the others I’d have to look up or open the files. I’m thinkin’ that process might be a little overwhelming with 120K tracks.

So I’m sittin’ on my couch getting ready to start. Got a network cable plugged into my laptop, rather than using my usual WiFi... and a stupid Windows Update runs and drags the performance down to a crawl. Just rebooted, so maybe this time I can actually do something.

I’m thinkin’ I’ll just get started... Do as much as I can each evening while sitting with the wife and the dogs, and we’ll get done when it’s done. Might take 10 years... I dunno???
 
Likes: ittigger

ittigger

Hundred Acre Industry Icon
Feb 1, 2011
15,078
11,666
115
Western Maryland
#9
Just started ripping ALL of my Top Hits, Promo Only, and Prime Cuts into iTunes folder on my external HD in MP3 format. ITunes is great for doing lists and printing those lists...entire library, if necessary. I will then import select lists into my digital software program OTS. iTunes also does a wonderful job of searching for duplicates.
As long as you're not ripping them into iTunes containers, then you're fine. I would also not keep them in the 'iTunes' folder - as this gets much closer to the 255 limit discussed in another thread. iTunes is great for creating play lists - but beware - too many lists will slow the machine down. If your directory structure and file naming is setup 'properly', you can easily create a 'catalog' through a command prompt and clean it up (make it presentable) in Excel or Adobe. 'Properly' is defined by you.

Anyhow, the reason for ripping my professional DJ source CD is so that I can pack them up for storage...maybe even sell. Regardless, I have no need for them any longer now that I have the external HD. Ripping them at 320 CBR. Long project. It will be months before I'm done. Like 52 weeks x 20 years.
I do the same - I rip the discs (also 320 CBR) and then they go to storage. According to Copyright law, if you sell the disc, you must also get rid of any copies (to include ripped mp3's) of that disc.
 
Last edited:

ittigger

Hundred Acre Industry Icon
Feb 1, 2011
15,078
11,666
115
Western Maryland
#10
I can see how having upper level genre folders could be useful, but it might be difficult to do, given the genre is in the MP3 tag. I probably know off the top of my head what the genre is for maybe half of the artist/albums, but the others I’d have to look up or open the files. I’m thinkin’ that process might be a little overwhelming with 120K tracks.

So I’m sittin’ on my couch getting ready to start. Got a network cable plugged into my laptop, rather than using my usual WiFi... and a stupid Windows Update runs and drags the performance down to a crawl. Just rebooted, so maybe this time I can actually do something.

I’m thinkin’ I’ll just get started... Do as much as I can each evening while sitting with the wife and the dogs, and we’ll get done when it’s done. Might take 10 years... I dunno???
MP3Tag - it'll take you seconds to put them into folders by Genre, as long as a given album has the same genre listed. The only issue I immediately foresee would be the 'Various Artists' discs but this could/might be easily avoided.

If you don't have the Genre listed in the tag, you'd have to put it in manually. I'm sure there is some automated program that could validate it against CDDB or something - but IMO, that's nowhere near as accurate as it should be.
 
Last edited:
Likes: DJ Bobcat

Proformance

DJ Extraordinaire
Nov 6, 2006
5,644
3,076
115
#11
I have some advice for you but, for the sake of those starting out or still in the early years let's look at how this really relates to doing business as a DJ. I am offering you a purely business perspective.

Based on the description of your effort and music library it's fair to say your primary job has been focused on IT and the information storage business. :) It's an important distinction, because if a DJ is not clear about the real job - then any bottle-neck in the reward stems from the misplaced effort.

The true job of ANY disc jockey is to: PLAY music. The following are not in the job description for a DJ:
  • Collecting
  • Server Installation & Maintenance
  • Structured File systems
  • Information and Content management
In a digital age it's important (especially for perfectionists) not to get sucked into your personal computer and lose sight of why we have the music, and the criteria that makes a song worth storing.

What to do now?

My suggestion is to leave it alone - it's a done deal. You've already noted that storage is cheap, and it would take forever to methodically undo this -So don't. Every DJ program has a search utility to pull up what you want.

Yes, browsing a library that big isn't very helpful if you're looking for direction, but you can browse online music sites to get more relevant and trending results while doing gigs.

Going forward, I suggest you delete files on the fly while working. Then delete those folders from the other locations and overwrite with the now reduced folders.

Many will disagree but, the true value of your music library today is $0.00 Remember this every time temptation leads you back into file management mode. Granted, we are all guilty of doing things like this because it keeps us feeling engaged and connected with our hobby or work - but, it's truly not productive.

For comparison, my collection is about 20,000 tracks with perhaps as much as 30-40% duplicates (other versions.) I've been doing this full time for 30 years across the full spectrum of of events and hit 90% of all requests I get. I was also a club jock, music buyer for a major hotel chain and some multi-ops. I know from experience that an active DJ collection need only have about 5,000 popular selections to meet that 90% satisfaction rate.

You should also take note that I use the word "collection" when I refer to my music set, and "library" when I refer to yours. A disc jockey doesn't need an entire library to do this job. We just need a really good collection of active or popular favorites.
 
Last edited:

rickryan.com

DJ Extraordinaire
Dec 9, 2009
14,996
11,062
115
54
Hendersonville, TN
www.RickRyan.com
#12
I'll take a different direction, DON'T try to rearrange/delete tracks. Hard drive space is cheap and as long as your player's search utility lets you find the tracks you need, then don't mess up what's already working. The only possible exception I can think of it duplicates. I believe there are tools that will scan and eliminate dups. Be sure to rebuild your player's library after doing this.
 

Proformance

DJ Extraordinaire
Nov 6, 2006
5,644
3,076
115
#13
According to Copyright law, if you sell the disc, you must also get rid of any copies (to include ripped mp3's) of that disc.
The people going door to door checking up on that are......?

FYI: That's not specifically in the Copyright law, though it's a logical assumption based on prior trade of physical media. As that physical media disappears this whole problem will essentially go away with advancing digital content protection.

Consider for example, the 120,000 music files in question - 115,000 of which probably haven't been opened or edited since they were created. He could donate all of those CDs to the next church yard sale and no one would care. :)
 
Last edited:
Likes: B-Sharp

DJ Bobcat

DJ Extraordinerror
Nov 8, 2014
6,389
9,534
115
Oklahoma City
#14
I have some advice for you but, for the sake of those starting out or still in the early years let's look at how this really relates to doing business as a DJ. I am offering you a purely business perspective.

Based on the description of your effort and music library it's fair to say your primary job has not been as a disc-jockey. An unusual amount of your effort has been focused on IT and the information storage business. It's an important distinction, because if a DJ is not clear about what his real job is - then any bottle-neck in the reward stems from the misplaced effort.

The true job of ANY disc jockey is to: PLAY music. Anything else you do is time that will not produce a worthwhile economic result from a potential customer. The following are not in the job description for a DJ:
  • Collecting
  • Server Installation & Maintenance
  • Structured Filing systems
  • File & information management
In this digital age it is important (especially for perfectionists) not to get sucked into the TRON world of your personal computer and lose sight of why you have any music in the first place.

What do you do now?

My suggestion is to live and work in the present rather than the past. Leave it alone - it's a done deal. You've already noted that storage is cheap, and it would take forever to methodically undo this. So don't do it.. Every DJ program has a search utility and you can easily pull up what you want. Yes, browsing a library that big isn't very helpful if you're looking for direction, but you can browse online music sites to get more relevant and trending results while doing gigs.
Going forward, I suggest you simply delete files on the fly while you are working. Then delete those folders or the entire libray from the other locations and overwrite with the folders or library you just edited. Maintain only 3 copies! One to play from, one as a mobile backup, and the last as a fail safe archive.

You're not going to like this - and many will attempt to disagree but, the true value of your music collection today is $0.00 Remember this dollar value every time you have the urge to fall back into file management mode and train yourself to go do something else instead. Granted, we are all guilty of doing things like this because it keeps us feeling engaed and connected with our hobby or work - but, it's truly not productive.

When we pass on, no one is waiting in line to take over our hard drive management; nor should they in an age where the entire world catalog is at their fingertips waiting to be streamed.
There’s some surprisingly good advice in there. Some of it I’d already concluded myself. I decided that DELETING FROM was much faster than COPYING TO. I don’t disagree with you on the value of my library (or anyone else’s library). However, I can’t see myself relying on any other source from which to play music at paid DJ gigs any time soon, so my library is of GREAT value to me. I’m gonna go ahead and clean up my DJ music library anyway, just because it will make it easier for me to maintain it going forward, and because... well... because I can.

I think we can all agree that MOST of the stuff I do (related to my DJ business) is NOT very productive (see my DIY threads).
 
Last edited:

DJ Bobcat

DJ Extraordinerror
Nov 8, 2014
6,389
9,534
115
Oklahoma City
#17
Speaking of this discussion, what are some good tools to get rid of duplicate MP3s?
Windows (10) has the capability built in... CCleaner. I’ll admit, I’ve never used it, but it’s something I’ll likely try while while doing this project. The reason I never did it before is I wanted to keep entire albums in tact. If you have albums by a certain artist, many of the same tracks will appear on their greatest hits or compilations. In my DJ library, only one version is needed, and I’d never need to play a whole album.


Sent from my iPhone using Tapatalk
 
Likes: ittigger

ittigger

Hundred Acre Industry Icon
Feb 1, 2011
15,078
11,666
115
Western Maryland
#20
The people going door to door checking up on that are......?

FYI: That's not specifically in the Copyright law, though it's a logical assumption based on prior trade of physical media. As that physical media disappears this whole problem will essentially go away with advancing digital content protection.

Consider for example, the 120,000 music files in question - 115,000 of which probably haven't been opened or edited since they were created. He could donate all of those CDs to the next church yard sale and no one would care. :)
It's called Ethics - of which you appear to display none with that statement. The tax man isn't going door to door either - should you not pay your taxes? You seem to like to try to change sides. You can't both enforce what the law says .. and then try to flaunt it.

Section 117 of the Copyright Act:
Under section 117, you or someone you authorize may make a copy of an original computer program if the new copy is being made for archival (i.e., backup) purposes only; you are the legal owner of the copy; and any copy made for archival purposes is either destroyed, or transferred with the original copy, once the original copy is sold, given away, or otherwise transferred.

-----

Technically, the answer is yes. Under general copyright law, distributing a copy of copyrighted materials (like a music CD) can only be done with the permission of the copyright holder (usually the artist or record label). Moreover, the No Electronic Theft Act, a federal law, states that it is a federal crime to reproduce, distribute, or share copies of electronic copyrighted works such as a music CD. This can be true even if you copy and then give the CD away without a commercial purpose or receiving financial gain.

The law creates an exception for "personal use," which means that burning copies is legal if the person plans to use the copies for their own personal use. Practically speaking, this means that once you have bought an album, you can burn it in order to play on your portable music player, computer hard drive, etc. The law, however, does not allow someone to burn a CD and then pass the copy on to others. Passing out burned copies of the CD to family and friends or otherwise giving away a copied CD is not considered "personal use" and would be in violation of federal law. The same general rules apply to other copyrighted materials like movies, video games, or software programs.
 
Last edited: