[WIP] MusicBrainz mirror on FreeNAS

Status
Not open for further replies.

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Yay!

Now to template your build and distribute it to the world.. mwahahaha!
 

9C1 Newbee

Patron
Joined
Oct 9, 2012
Messages
485

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
chris67,

Do you plan to put together a series of comprehensive steps for this? If not would you be okay with making this into a template for the rest of us?
 

chris67

Cadet
Joined
Aug 23, 2014
Messages
9
Well, I'm happy to do either but like I said, I think there's a few bits and pieces to iron out first - the redis server error, running the plackup command at startup (and under a user other than root) and setting up replication. I'm not a complete novice with freebsd but my background is MS Windows so no guarantee of success with any of that.

The other issue is that the jail is 8.5GB in size after importing the database. And the import was from files I downloaded weeks ago when I first started looking at this. So I suppose the answer is to drop/delete the database and then create the template. It would be down to the individual to import the latest copy of the db. In the meantime (and when I get a spare hour or two), I might create a fresh installation and make sure I document every step as I'm not sure whether I followed previously posted instructions faithfully or not.

Any thoughts/advice on any of the above welcome.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Just a few pieces of advice/questions:

1. When you say 'replication' what are you referring to?
2. To be honest, running as root in a jail isn't as big of a deal as it sounds like. If "someone" were to get "root" access to your jail they'd still be stuck in the jail. Not much gained there.
3. Do you know if the database compresses at all? Normally templates are in tar.gz so compression is used. I figure it "should" compress well because the appliance VM that is given out has a database in it and the file is less than 2GB.
 

chris67

Cadet
Joined
Aug 23, 2014
Messages
9
Just a few pieces of advice/questions:

1. When you say 'replication' what are you referring to?
2. To be honest, running as root in a jail isn't as big of a deal as it sounds like. If "someone" were to get "root" access to your jail they'd still be stuck in the jail. Not much gained there.
3. Do you know if the database compresses at all? Normally templates are in tar.gz so compression is used. I figure it "should" compress well because the appliance VM that is given out has a database in it and the file is less than 2GB.

1. Replicating changes to the local musicbrainz database.
2. OK, that's useful to know, thanks.
3. I'll test that when I have a working jail. However...

I think I've made a schoolboy error which means I'm going to have to start the whole database import again. joelmusicman's instructions include modifying the DBDefs.pm file and changing REPLICATION_TYPE to RT_SLAVE. I did that but forgot to remove the # from the beginning of the line. Running the command 'carton exec -- ./admin/replication/LoadReplicationChanges' which should update the local database (https://github.com/rembo10/headphon...ping-your-local-musicbrainz-server-up-to-date) produces errors including 'relation "dbmirror_pending" does not exist'. I'm guessing the database wasn't created with everything needed to be a slave. Ho hum.

So back to the drawing board but not until tomorrow night. I might start again from scratch and, like I said before, make sure every step is recorded.

Incidentally cyberjock, you mentioned previously that "As for the allow.sysvipc=1 I have the fix for that." That would be useful as the jid is changing everytime I restart the jail.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526

chris67

Cadet
Joined
Aug 23, 2014
Messages
9
Thanks, that'll do nicely.

Still trying to create a build which just works. How can it be so difficult? It's like pulling teeth :eek:

Don't hold your breath.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
damn.. I saw your username as the most recent poster and I was like 'zomg.. here it is! here it is!'
 

chris67

Cadet
Joined
Aug 23, 2014
Messages
9
OK, better news. I abandoned my plan to rebuild and document - I kept getting "Bailing out the installation..." errors during the carton/dependencies install. After sobbing for a bit, I reverted to the jail which was working but was not configured as a slave. Ha. Easy to resolve. Corrected DBDefs.pm contents, dropped the musicbrainz database and reimported. I'm currently replicating changes from the online database into the local copy and I've set up a cron job to do so again in the early hours of the morning. Once I've checked that worked ok tomorrow morning, I've just got the "Redis server appears to be down" error on the web page to resolve (or do I? what exactly does that do?) and then... template time.

Just a point. I'm running freenas on a system with pretty minimal resources (sorry cyberjock, I know what you think about that!) so I don't risk ZFS. Is there an issue with creating templates from a jail on a UFS volume? I'm assuming not as it's just a tarred copy of the jail's files. Or am I wrong?
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
No, doing it from UFS shouldn't matter.
 

chris67

Cadet
Joined
Aug 23, 2014
Messages
9
Thanks. Another question which I can't find an answer for. I've created a jail on freenas with root password = "abc" so the jail has root password "abc". Now I create a template based on this jail. Someone else with freenas root password = "xyz" creates a jail using my template. What is their jail's root password?

Point being, I wouldn't want to distribute a template with my password "embedded". Although thinking about it as I type, that's probably not the case. Does the jail inherit the host system's root password?
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
if they create a jail using your template it will be whatever you set it to. So in your example the root password will be "abc". :)

Literally the template will be everything exactly how you set it. :P
 

chris67

Cadet
Joined
Aug 23, 2014
Messages
9
Thanks for the clarification.

Well, I think I've done as much as I can. Replication works and ran last night as a scheduled task (sorry, Windows background showing there). Sorted out running the plackup command at jail startup. Changed the root password to 'musicbrainz'. The only thing I can't resolve is the "Our Redis server appears to be down; some features may not work as intended or expected." message on the web page. I've installed the redis package but that makes no difference. I don't *think* it's important as it seems to be related to multiple sessions but please correct me if I'm wrong.

Currently running a final replication before I create a template (or try to anyway). And then... um, not sure. What's the usual method of making a template available to others? Signing up to a free online storage provider? Any recommendations?
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Yay! If you PM me I'll find someplace for you to send me the file. I think my ftp is up.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Ok gents. Template is uploading to me. Gonna do some checks and such and then find a place to host it and provide thorough instructions. :)
 

neto_hugo

Explorer
Joined
Mar 6, 2012
Messages
87

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Unfortunately, no. I'm wrapped up with 9.3 stuff right now (In fact, I'm watching my Mini throw up all over itself) while writing instructions for BIOS updates and stuff for FreeNAS Mini owners.

Hopefully this weekend I can get to this.
 

tonyschuite

Dabbler
Joined
Oct 8, 2012
Messages
12
ok, WOW! this thread has had way more attention than I have realised. I just started trying again and found your comments to be awesome!
however, I have found the guide mentioned above as well and will start to see if that would be the easier route :)

apologies for not sending replies, I apparently get hardly any notifications about this post.
 
Status
Not open for further replies.
Top