Backups finally sorted out

After many attempts with many different utilities I finally got the backup situation sorted out. None of the utilities I tried worked due to my custom configurations and Hard-drive setup. Ultimately, I had to resort to using the tar archiving utility built into all Unix systems.

Now I have to sort out my cron jobs and find a way to setup automatic incremental backups for both the system and the web site using tar.

Securing the Server....

After doing some reading I have decided to take the server back offline and keep it on my local network for security testing. I will put it back online once I feel confident about it's security.

Ubuntu and Joomla permissions issues

I've been dealing with permissions issues between Joomla and Ubuntu ever since I started this a week ago. I finally found a good guide so I'm posting it here.

The important bits are under the section heading "Configure Access Rights".

Posting it here for future reference:

https://help.ubuntu.com/community/Joomla

Edit:

Ok, so the permissions set-up from that link isn't completely correct as far as adding templates, components, etc to Joomla.

This will be a running list of the changes I had to make in addition to what is shown on the site that I linked.

1. During the pre-installation check a red flag was thrown stating that configuration.php was un-writable. This was because the file did not exits. I created a blank file named configuration.php and changed its ownership and permissions in order for it to get the green light for installation.

sudo touch configuration.php
sudo chown www-data:www-data configuration.php
sudo chmod 644 configuration.php

2. When installing the Kunena forum module I received several errors due to permissions issues and had to do the following to fix the problems before installation was possible.

sudo chown www-data:www-data ./tmp
sudo chown -R www-data:www-data administrator/
sudo chown -R www-data:www-data plugins/

Fortunately, these changes allowed the installation of templates with no further changes needed. Unfortunately, I am unsure about the sites security now. I will ask the Joomla devs about it.

Up and Runing

Well, I've managed to get it all up and running. Let me see if I can remember all I did though (should have written this as I went, but momentum took over).

For the MySQL setup, it turns out that the MySQL user, 'root', did have all the passwords setup from when I installed the LAMP stack, which means it is secure. While that was a good thing, it did not have any permissions at all, which is not the way it went last time around. The solution was simple enough. I had to create another user with all permissions. Once that was taken care of I created a new database for the joomla install then made a new MySQL user named Joomla which I gave all permissions to for that database, but not for MySQL as a whole. That seemed to work out nicely.

Another issue I ran into that didn't happen before was that my 'configuration.php' file could not be written too by Joomla during the install. I'm not sure what a good long term solution to this would be, but for the time being I just changed all permissions to the file to '777' for the install then back to '644' afterwards. The problem with that is I will have to change it back to '777' every time I want to change the global settings then back to '644' for security reasons. I don't know if this will cause any issues while working on the site...will have to wait and see.

Configuring MySQL

Unfortunately, my memory being what it is, I installed the LAMP stack 2 days ago and now I cannot remember if I setup a MySql user account or not. I somewhat remember setting up a password for root@localhost but don't remember if that applies to root@127.0.0.1 or not.

For the time being, I'm going to install one of the GUI's for MySQL since I really don't want to have to do everything with the terminal. Then it should be easy to see what has already been done and what is still lacking.

Decided to use mysql-admin since I have only ever used phpmyadmin and wanted something new. Currently doing system security updates so I'll finish this post and come back to it later.

Hopefull resolution to joomla permissions problem

I previously had problems with Joomla's site permissions in Ubuntu so I went ahead and reconfigured apache2 to use new config files and a new base directory for Joomla. I moved it to the user accounts home directory and deactivated the default site directory along with the settings in the old config files for apache2 and enabled the new directory and settings then restarted the apache2 server to enable all the changes. So far so good, but I won't really know anything until I finish configuring the database server and get Joomla installed.

This was all done according to the Ubuntu docs found here:

https://help.ubuntu.com/community/ApacheMySQLPHP

Finally, I moved index.html and test.php to the new home folder to test it all out...it works like a charm.

A note about backups.

Just a reminder to myself that I need to find a better way to do backups for the server. I thought I had gotten ahead of this problem on the first install and partitioned the hard drive to have somewhere to safely store my incremental backups. I did this using dd_backup but ran into several issues and ultimately could not use it to restore the system following my previously mentioned problems.

Mondo Rescue looks like it could work, although I will have to find a way to mount the backup as a virtual dvd drive or something because I really don't want to spend several hours a week swapping out dvd-rw's for a backup.

I will need to look into this further. Until then, I will just have to be careful and accept that I will have to do a full reinstall if I screw it up again.

After doing some tests to see if the apache2 php mod was installed I found out it already was, just wasn't working for some reasons. I restarted the apache2 server service and it now works...unfortunately, now the server is crying about not having a fully qualified domain name.

Sigh...I really want to ignore this and deal with it later, but I know I will have forgotten all about it by the time it becomes an issue and be jumping through hoops to try to figure it out. Its not really a problem but better to do it right the first time...So I will setup the aliases for localhost and 127.0.0.1 and be done with it.

Apparently Ubuntu's idea of a pre-packaged LAMP server does not include the apache2 library mod for php...makes no sense at all, but no use complaining.

Using "<?php phpinfo(); ?>" to test the php functionality didn't display the page, but tried to download the file instead. I will have to manually install the php mod lib.

Current project: Ubuntu with Apache2, Mysql, and PHP. Followed by a fresh Joomla install to run a website for my son's private Eve Online Incursions group.

In all honesty, I really want to do this with a Python CMS instead of PHP. Unfortunately, the learning curve is too steep to be able to do it in just a couple of days with no previous experience and I told him I would have it running for the weekend. So, here we go...

Fresh install of Ubuntu 11.04...


After learning all this the hard way (installing and configuring each app independently), I've discovered a program called tasksel and decided to use that to install and do basic configuration for a LAMP server.

So far so good. apache2 configured and working properly...now to test php functionality.

Deja Vu all over again...

Wow...6 years since my last real foray into Linux, server building, and web development. I guess I shouldn't feel so bad then that my first attempt in 6 years has ended with recursively removing my entire system. This is the sort of thing only the most inept computer users have been able to accomplish. I, however, did it with 'sudo rm -Rf ./' ... meaning I couldn't get it to work so I threw in some flags that BRUTE FORCED the recursive deletion...even the greatest programmers can't stop me from complete system deletion with all their fancy blocks and safety features in place.

To be fair, I thought I was in /var/www instead of the ROOT FOLDER! Anyways, live and learn I guess. Its amazing how the things you kid others about winds up happening to you, even 6 years later.

Onwards and Upwards!

In light of the aforementioned error, I decided to start making a log of my builds and configurations in order to have an ability to review the sequence of events leading to my eventual failures...then it hit me, this all seems too familiar. I must have done this before. A simple search through my old accounts and...voilà!

So, its deja vu all over again...wish me luck.

Onward and Upward...

Well, I spent a month or so with gentoo and after all that time compiling, configuring, and prelinking I wound up nuking all my main config files after updating them following an emerge-world... live and learn.

I did learn quite a bit from the gentoo experience; however, while working with gentoo I began to question whether all the time spent compiling, configuring, updating, and prelinking was really worth the increase in speed you get from "total optimization". So I spent some time on #gentoo talking to some devs and my suspicions were pretty much correct. The increase in speed really is pretty minor (very negligible actually) when you consider the time spent to do it all. So I started my search for the fastest 'out of the box' binary distribution and found the following distros that met my requirements enough to justify a trial run.

Puppy Linux

Supposedly the fastest distro out there. This is set up to run in a ram drive, so it has to be the fastest. Unfortunately, I didn't actually test it out because of the lack of packages available for it...whats the point of being the fastest if you can't do anything with it but surf the net and use Open Office and other basic tools? It is a rather interesting setup though so it is worth keeping an eye on just to learn the technology of their methods (which is the only reason I even mentioned it here).

Yoper Linux

I ran this for a couple of days. The installation goes without a hitch (as long as you know your hardware setup) and was my first choice due to it being optimized for i686 systems. My goal here was to simply install, do prelinking, and see what she could do. Unfortunately, after installing the system froze while using apt-get (by way of Synaptic) to update the system. Since installation is relatively quick and easy I just reinstalled and tried again; the same thing happened though. It seems to be an intermittent problem during the update but could be a server issue with a particular repository. I was going to look into it further but was thrown back when I jumped in #yoper and found a whopping 6 people on the channel. I posed a few questions to the channel which no one replied to; then a new guy showed up and remarked on how surprised he was to see so many people on the #yoper!!! (no, he wasn't being sarcastic, I asked)... Anyway, after talking to him for a while I learned that Yoper is completely run by one person and is going to a fee based update system. I immediately booted up to my windows drive and began downloading Ubuntu.

Ubuntu Linux

I've spent the last few days now with Ubuntu and have been very surprised at its quality and attention to detail in how this system was designed. The organization among the devs is is amazing and is clearly shows up in the distro. So far I have installed, configured, updated, and added to this system without running into a single snag along the way. With Ubuntu you get the ease of use comparable with SuSe and Mandrake, but without the bloat. I haven't done any comparison testing for the speed of this system yet but so far the only slow downs I see are the usual slowdowns you get while opening large applications like Open Office (after I prelink the system that won't be a problem).

Over all, I would have to say that I'm pretty much back at square one by going back to a binary Linux distribution. I learned a lot in the last month while doing the gentoo thing and I will certainly go back to gentoo in the future. If Yoper could get its act together and join the open source community rather than trying to focus on its future potential to make a little money I would definitely go back to it. Unfortunately, I don't see this ever happening. On the up side though, If I stick with Ubuntu for more than a month I will definitely start working on an i686 optimized version of it as a contribution to the Ubuntu community. On top of all the other good points with Ubuntu, it probably has one of the most organized and active communities I have seen so far.

To cap it all off, I'll list all of the distros I have tried over the past few years - without listing them in any particular order (other than the order I think of them in):

RedHat

Mandrake

SuSe

Fedora Core

Knoppix-STD

PHLAK

LFS 5.0

Gentoo from all three tarball stages

EnGarde

SLES

Mepis

Yoper

Ubuntu

Linux Adventures

After 6 days and 4 seperate build attempts (1 stage 1, 2 stage 2, and 1 stage 3) I finally have a working system, albeit a minimal one. Currently, this stage 2 build is in its infant stages and has 2 major issues that need to be resolved before continuing on this journey:

1. DMA must be turned on for the HDD if the system is to run efficiently. As of now, I do not know if this will require a recompilation of the kernel...but I will do that if necessary.

2. For whatever reason, my network configuration will not start up on boot. So far, I think that
the problem may be due to not having the proper network driver compiled in the kernel. However, it may be possible for me to simply add a module to the kernel without having to recompile the kernel. Having said that, I will more than likely recompile the kernel anyway in order to fix the DMA issue. If so, I will just add the proper network driver at that time. For reference, I give the errors here:

Bringing eth0 up (192.168.1.101)...
SIOCSIFADDR: NO SUCH DEVICE
eth0: Unknown interface: No Such Device
SIOCSIFNETMASK: No Such Device
ERROR: Problem Starting Needed Services.
"netmount" was not started.

Possible solutions to these problems:

1. I am considering recompiling the gentoo-dev-sources kernel with the genkernel application with a switch that will allow me to enter into ther kernel configuration setup and add the changes. The reason I am thinking of using the genkernel application this time around is that kernel optimazation is not my strong suit at the moment. Until I find better documentation I simply do not have the time to do this by trial and error. There are simply too many diferent configuration options that are not based on hardware descriptions as I know them. Being a hardware guy by profession, it is not easy for me to accept this fact. After I have a base system fully functional, I will deal with this issue while I am learning how to build my "Perfect" system.

2. Start from scratch with a stage 1 tarball so I can have a fully optimized system from the boot strap on up. Since this is a learning process this isn't such a bad idea; however, due to time constaints, I will attempt to recompile if I cannot resolve these issues through manual configurations first.


Current concept of my "Perfect" build:

I want a completely optimized build from a stage 1 tarball that will be flexible enough to allow for different configurations. I'm not entirely sure about how this will/can be implimented yet. I know it is possible to simply compile different kernels that will be optimized for different tasks. i.e. one for development, one for desktop, one for servers, and one for gaming. This is all well and good; however, it is not as simple of a system as it probably can be. Having to reboot the system into a different kernel/configuration everytime I want to switch modes is hard on the hardware and inefficient. A better solution may be to compile one kernel with all of the necesary settings for each of the tasks, then create different user accounts that hold the configuration settings for which ever task is at hand. Unfortunately, I don't really see how this one kernel can be optimized for each of these tasks without rebooting...more on that later.

Just for reference, I am going to throw in a history of past problems with my builds so I can have something to go by in the future.

1st Build Attempt - Stage 1 Tarball:

Given my past experience building Linux from scratch, I was really excited at the possiblilities that are available with a Gentoo build. After having built Linux From Scratch 5 times with only 1 real success, an 'optimized from the bootstrap up' Linux OS with an awsome package management system that is designed around compiling each package for optimizaiton had my adrenaline pumping--Yes, I am a complete Geek. This first experience with Gentoo was a slow process that consisted of more reading than anything else; the type of reading that puts most people to sleep. In my case, however, I found it informative yet perplexing at the same time. The biggest problem I found with the Gentoo book is that it is more of a step-by-step guide than anything else. Even though there is a 'non-verbose' version that really is nothing more than a step-by-step guide, I would have liked there to have been more of an explanation of how the things I was doing matter. Also, the book always states that it always gives you the option to make the system the way you want it, but it does not go into enough detail about what those options are. A better explanation of your options and a more thorough guide to the way Gentoo's system works would be welcome.

From what I remember about this build (it was 6 days and 4 builds ago), there were a couple of dependency problems when I tried to 'emerge system'. From the #gentoo irc channel, I found that there were some problems with the latest version of gcc but I do not remember this being a problem for me during the build. In this and my subsequent build attempts, the dependency problems I ran into were due to too much optimazaiton on my part. I had somewere around 40 or 50 optimazations set in my 'USE=' flag in make.conf. NEW RULE - ONLY USE THE NEEDED OPTIMIZATION SETTINGS IN make.conf! I should have known this from my previous Linux From Scratch builds, but my memory SuX0rS. Anyway, I digress...

After rebooting the system on my first build attempt, it did not boot. First of all, for whatever reason the graphical interface at the Grub screen was so bad that I couldn't read the text and non-alphanumeric characters showed up as blocks. I did not figure out what was causing it yet but I do have a theory or two. I didn't really sort this out until my third build so I will save the fix until I get to that part. After hitting Enter at the Grub screen I received an error about it not being able to mount the boot partition. I searched the forums and hung out in #gentoo for...too damn many hours, but I did find someone who had the same issue. It called the partiion 0x7,
which apparently is NTFS. At the time, I had Windows on hdd0 and was building Gentoo on hdd1. I anticipated problems before I started all of this so I had turned off hdd0 in the BIOS before I started. LESSON LEARNED - THE LINUX KERNEL IS SO POWERFULL THAT AFTER BIOS PERFORMS POST, LINUX DIRECTLY ACCESSES THE HARDWARE AND ONLY USES THE BIOS CONTROLS THAT YOU TELL IT TOO WHEN YOU CONFIGURE THE KERNEL BEFORE COMPILING IT!

Anyways, I lucked out on that one and after trying, and failing, another build from scratch with the exact same two problems, I physically disconnected hd0. As a side note, I had manually configured the bootstrap on both of these attempts so I am 100% positive that it was (suposed to be) pointing to the partition on hd1.

2nd Build Attempt:

This one went alot faster having skipped the bootstrapping compilations, but the result was identical to the first attempt. Just as in the first attempt, the second attempt had some dependency problems on 'emerge system'. Also, since I failed to mention it before, both the 1st and 2nd builds were completely manual kernel builds.

3rd Build Attempt:

After the first two failures, I decided that I'm an idiot and need to do the easiest build possible. This build was a Stage 3 Tarball using genkernel with defaults. This has to work right, wrong. This time around I decided that if nothing else, I was going to resolve the problem with the graphics. In both previous attempts, I compiled the kernel with framebuffer support with vga and vga-(?). Framebuffer is old technology but it sure makes your boring old console look nice. I bet my father would have loved this if it were available back in the IBM 8088 pre-XT days. On top of framebuffer support in the kernel, I compiled and added the nvidia-kernel module from
portage. I figured it had to be a conflict so on this 3rd build, I did not compile the kernel with framebuffer support...it resolved the issue. Unfortunately, I really want framebuffer support, but I also really want to try out linux as a gaming platform, which means I absolutely, without a doubt, HAVE to have both of them. I'll let you know when I get this sorted out. ::Digress much?:: Anyways, after reboot I got the infamous 'cannot find file' error. After many hours and three different attempts at recompiling the kernel in a chroot envirenment, I finally fixed it. Unfortunately, another problem popped up and it still did not boot.......I fixed it. Then another problem pre-empting boot...fixed. Then the second boot dropped. The reiserfs file system on the root partition was corupt and the supper block was no wheres to be found. So I did what all good Linux folk do, I said FSCK IT! I ran reiserfsck --rebuild-sb but it didn't fix it. Then I jumped through all the hoops I could find and wound up running reiserfsck --every-available-option-ther-is...and I finally repaired it without any data loss. Unfortunately, filesystem problems kept creeping up before I ever got fully booted.

In the end, I had to do what I had laughingly told thousands of idiots who call tech support because they are infested with viruses, trojans, and worms -- Oh My. I ran debug, then did a low level format which ran through out the night...

Stay tuned for more adventures of 'The Linux Zone'.

What is your personality type?

Ever wondered who you really are? After a lifetime of compliance and conformity (or lack there of) it is often difficult to really know who you are at the core of your very being. Take the Jung personality test to find out. After submiting the test, click on the links for the description of your personality type.

Mine is INFP. Read about it Here and Here. I'm not sure every part of the description fits my personality, but its still close. I wonder what the results would be if I took this while in a different mood...

-------------------------------------------

Took the test again today (Oct. 01, 2004) to see if the outcome would change based on my mood; it didn't. I changed quite a few answers but the only thing that changed was the distribution. I'm still INFP. The first time I took the test I came out as 33, 11, 11, 11; this time, it was 78, 11, 44, and 33 (thats Introverted, Intuitive, Feeling, and Percieving respectively).

Go ahead and take the test yourself by clicking on the title of this post (almost all of the titles on these blogs link to external sites), then come back here, click on "Comments" at the bottom and tell us your results.

------------------------------------

Took the personality test again today (10/31/04).

I have been bothered lately because it seems I have not really followed along a path that I should be following if the test is correct. So, I took the test again and the results follow:

Your Type is INFP

Strength of the preferences %

Introverted 89
Intuitive 33

Feeling 11
Perceiving 22

INFP type description by D.KeirseyINFP type description by J. Butt

Qualitative analysis of your type formula You are:
very expressed introvert
moderately expressed intuitive personality
slightly expressed feeling personality
slightly expressed perceiving personality

~SIGH~

Looks like some soul searching is in order - perhaps this explains my constant condition of unhapiness and general dissatisfaction in my life...