One of the most important factors that will lead to visitors leaving is slow loading pages. And at that point, we should make distinction between sveral types of sluggishness: A. Slow due to HTML bloat B. Slow due to long server processing time C. Slow due to long browser rendering time.
My hard drive on the old server were almost full. The postgres database grew to be almost 100GBytes, and I had to do something otherwise an anticipated server crash is on the way. The solution I had in mind is to use the old server, which already has two 150GB hard drives. Making one volume using the two drives can be accomplished in several ways, the easiest way is to create a logical volume out of the two. The other way is to make a software RAID 0 array. In RAID 0, the kernel stripes the data across the hard drives, so a sector gets stored half on each drive. This in theory should lead to twice the throughput, however SATA and IDE are serial devices so the bottle neck will be how fast is your SATA or IDE device.
So it took me a while to set up the RAID 0 array on the box using UBUNTU and the reason is that we have to use the server installation CD. The GUI desktop CD does not have the RAID option - or at least I wasn’t able to figure out where to find it. On the server CD, the easiest way is first to create two identical partitions on the two hard drives, and select the partition types as RAID. An option will then appear that says something like “Create a software RAID device”. You can use that to include both partitions in the RAID device. Remember that you must have a small /boot ext3 may be 100MB, and swap partitions.
There was an obvious increase in performance over a single drive, although I did not measure before and after but from what I see the intensive hard disk operations that used to saturate the drives when monitored by atop no longer does that. My interpretation is because the peak bandwidth of the SATA controller is larger than any hard disk by itself plus we have buffers in every hard disk, so effectively we’re using twice as much buffers.
Moving the database and websites took almost all the night, I started pg_dump on one machine over ssh, piping on the other side in a psql process. I stayed two hours after I started it, then I gave up and went to sleep. In the same time I had processes running to sync the files and my subversion repository.
As usual, I saw the update icon in Ubuntu’s taskbar so I ran apt-get update and apt-get dist-upgrade, and as I read I found an update for freenx. When dpkg was installing the newer version it said that the configuration file node.conf is different. I pressed D to run diff on them to see whether the old problems were solved or not. I believe this package should be bug free almost for every one.
The package sources I’m using are:
deb http://www.datakeylive.com/ubuntu gutsy main
deb-src http://www.datakeylive.com/ubuntu gutsy main
Since I updated the website this weekend and it’s been downloading images from MRIS non-stop. In a previous post I said that I had around 900,000 images. I later on discovered that I had dangling images whose listings were deleted. Since this weekend, the update script downloaded almost 820,000 images from MRIS and inserted those binaries inside bytea fields in postgresql. I was expecting things to slow down, however I don’t feel any slow down since the process started. It seems the postgres team really optimized the b-tree indexes to the extreme.
Ok… I was trying to do this in my home network, and for the very first moment any one trying to do this will think about using a samba share. There’s another way, which is much simpler.
Set up the cups server and allow internet printing from your cups server. Go to your windows, add a printer and use http://IP-address-of-linux:631/printers/<PrinterNameInCups>
That’s it ! choose the correct printer driver from the windows drivers, mine was HP1100 LaserJet. It worked flawlessly with on troubles.
On the windows side I’m using XP and the linux side is ubuntu gusty.
It’s really bad. I really feel bad from what Microsoft is doing, locking vista if they felt any change. There should be a better way to verify their licensing. They should basically lower their “false positive” rate as it looks like they’re hitting innocent people as well.
Yesterday we had a power surge at home. I noticed around 10 am when the website was not responding. When I went back home, I found both the webserver and my gateway machine both having errors and ubuntu halting at “run fsck manually”. Luckily, I was able to fix the webserver without a re-install. I have keep daily backups from both my postgres and mysql databases by dumping them around 4:00 am daily. The dump is stored on a separate hardrive, which gets unmounted after the dump completes.
I had to re-install the gateway machine. On the good side I got to re-install gutsy instead my older feisty installation. No data lost, as I keep my /home always on a separate partition, I only had to re-install the OS.
To prevent future damages and save time, I bought a UPS 900KVA , which should be fully charged when I go back today. Also, I got a KVM switch, instead to moving the monitor, keyboard and mouse connections over the four machines.
On one of my searches on Google zaf’s blog came up again, this time he was complaining from SSH attacks. I looked in my /var/auth.log and I find this:
Aug 21 13:56:17 local sshd[7744]: Failed password for root from 202.171.152.211 port 42343 ssh2
Aug 21 13:56:19 local sshd[7752]: (pam_unix) authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=202.171.152.211.static.zoot.jp user=root
Aug 21 13:56:21 local sshd[7752]: Failed password for root from 202.171.152.211 port 43304 ssh2
Aug 21 13:56:26 local sshd[7768]: (pam_unix) authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=202.171.152.211.static.zoot.jp user=root
Aug 21 13:56:28 local sshd[7768]: Failed password for root from 202.171.152.211 port 44577 ssh2
Aug 21 13:56:30 local sshd[7770]: (pam_unix) authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=202.171.152.211.static.zoot.jp user=root
Aug 21 13:56:31 local sshd[7770]: Failed password for root from 202.171.152.211 port 46364 ssh2
Aug 21 13:56:33 local sshd[7776]: (pam_unix) authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=202.171.152.211.static.zoot.jp user=root
Aug 21 13:56:36 local sshd[7776]: Failed password for root from 202.171.152.211 port 47340 ssh2
Aug 21 13:56:38 local sshd[7782]: (pam_unix) authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=202.171.152.211.static.zoot.jp user=root
Aug 21 13:56:40 local sshd[7782]: Failed password for root from 202.171.152.211 port 48644 ssh2
Aug 21 13:56:42 local sshd[7790]: (pam_unix) authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=202.171.152.211.static.zoot.jp user=root
Aug 21 13:56:44 local sshd[7790]: Failed password for root from 202.171.152.211 port 49725 ssh2
Aug 21 13:56:46 local sshd[7800]: (pam_unix) authentication failure; logname= uid=0 euid=0
tty=ssh ruser= rhost=202.171.152.211.static.zoot.jp user=root
Aug 21 13:56:47 local sshd[7800]: Failed password for root from 202.171.152.211 port 50548 ssh2
By just putting the IP on Google, I found that this guy is very famous. And it repeats twice or three times a day from another Korean IP. After searching the net the most promising and easiest to install solution was denyhosts . It looks in the /var/auth.log and applies rules to filter the IPs which are attacking your machine. It then adds them to /etc/hosts.deny . A nice feature is a shared xml/rpc service where every host running deyhosts can share the IPs trying to attack his/her machine. Also, download the latest black list .
Installation was practically trouble free. I used synaptic to add the package, edited /etc/denyhosts.conf and enabled the SYNC parameters to share the IPs of attackers. After enabling the SYNC, my hosts.deny was filled by more than 1400 IP addresses ! Amazing how much time those people have to annoy us, instead of focusing on building something useful.
A quick not - I noticed that the attackers recognize services running on standard port numbers. So they assume that FTP is on port 21, and SSH is on port 22. A very simple countermeasure is to run the services on non-standard ports - this would at least thwart almost 99% of those attacks.
I’ve read a lot in the internet, and many posts claim that freenx installs seamlessly and works without any hiccups even the ubuntu community docs, which might in fact have been true at one point of time. The one that really worked well with no problems is nxserver from nomachine, however, the free version of nxserver is limited to two users and two sessions. This makes it impractical if a machine is being accessed by more than two.
Freenx comes in as the solution however, every time I try to install freenx I spend at least an hour trying to guess what’s going on and then I leave off to something more useful. This time I decided to take it all the way to the end. I debugged every call from the nxserver file, till I located the problem and finally got it to work.
I will describe what I did and how I got it to work, although the solution might not work for every one and on every setup, yet it is better than nothing.
STEP 1
Add the following lines to your /etc/apt/sources.list [exactly like the community docs say]
deb http://free.linux.hp.com/~brett/seveas/freenx feisty-seveas freenx
deb-src http://free.linux.hp.com/~brett/seveas/freenx feisty-seveas freenx
STEP 2
apt-get install freenx
STEP 3
vi /usr/lib/nx/nxnode
Goto line 482 (press 482gg)
Edit the line (press i), change it to:
PATH=”$PATH_BIN:$PATH”
Add another line after:
$PATH_BIN/nxagent :$display 2>&3 &
Go out of editing mode - press ESC
Save the file type :wq and press enter
STEP 4
vi /etc/nxserver/node.conf
go to line 70 (70gg)
uncomment ENABLE_SSH_AUTHENTICATION=”1″
STEP 5
Download the nxclient for windows from the nomachine’s website. Make sure the SSL is not checked (i.e. SSL encryption is on). Connect with your client to your server and enjoy.
What was the problem with freenx then ?
The nxagent does not run as expected, and goes out with exit code 1 because it can’t find an option file somewhere inside the .nx directory. Earlier in the process, that option file should have been generated but it looks like it does not. After the nxagent terminates, your client does not find any one to communicate with on the other side and finally gives up and generates an error. If you remove the options parameter from nxagent line, sometimes nxagent goes out with an error because it can’t find a font ! so I ended up removing all parameters except the display. Well, it worked ok for me & I hope it does for the rest of you.
At last the nx server[/nx] is now working on my [tag]ubuntu box. I’ve been trying to get this running for a while now, and finally I discovered my mistake. I had X11Forwarding enabled in my sshd_config , which conflicted with the X11 forwarding the nxserver trying to do ! When I disabled it, all worked fine great and smooth ! Actually the error was the .Xauthority and .Xdefaults files in my home directory. A simple check, try to create another account and see if nxclient will successfully log into it and display the gui.
The picture on the left is a screen-shot of gnome with firefox running displaying google news.
The installation steps:
Downloaded the nxserver, nxnode, nxclient .deb pacakges from the nomachine.com website, and installed them using dpkg -i . I used the default key. I had to copy authorized_keys2 to authorized_keys because the default configuration of sshd assumes that the key is installed in authorized_keys and not _keys2 . The deb packages creates an nx home directory in /usr/NX/home/nx inside of which there is .ssh directory.
In one of my previous posts I described my frustration after hours of trying to make ecb (emacs code browser) to read php tags.
Today, I wanted to try the newest emacs version with gtk enabled. After downloading it, I compiled it on my UBUNTU machine with every thing enabled during the configure step (gtk, png, jpeg …etc) and I installed it in /usr/local just to make sure it won’t mess up the rest of my installation.
After playing with it for a little bit, I discovered that they embedded speedbar with the release, which was great. When I downloaded and installed php-mode and this time everything worked seamlessly !! speedbar now communicates with php-mode and exchanges the required tags.
Note that I also installed color-themes.el, I wasn’t able to find its website so I downloaded the file from the ubuntu website here.
So now I’m using emacs totally in development both in C/C++ and for PHP. Still I use VIM from time to time specially for editing system files.
I’ve been playing lately with emacs trying to get more productive using it. The GTK and X versions do not understand the DEL key ! the solution is to add (normal-erase-is-backspace-mode) in your .emacs file.
There are two scripts that I always use with VIM:
1. Tlist (or tags list): It lists the functions, variables in all opened source files.
2. Nerd commenter: Provides easy comment to source code, the easiest way is using Ctrl+C in during insert.
I’ve been using vim for a while, and lately I tried to use emacs for editing my code. I discovered that emacs was a much better editor, that takes so much time to configure. It is highly configurable to the extent that it can take you days to customize it to the point that makes you happy. I liked the M-; fast comment shortcut in emacs, and its C-style indentation (corresponding to set cindent in vim)
Since I was trying to emulate the same environment I had with vim, I looked for something similar to Tlist, and I found an elisp program called ecb (emacs code browser), the provides that tags in the opened source file. I also found the speedbar from the cedet-tools website, that provides a similar functionality. Both tools were not able to browse php code, unfortunately.
Although I believe that emacs is a much better editor, at the moment the available plugins do not support PHP, which for me is very important as I do a lot of php editing. However, for C/C++ I will be using emacs (or xemacs) for developing as it is much easier to work with.
Stallman’s why details a lot of reasons why would one release a free software under GPLv3. One reason is thwarting Microsoft’s future attempts to collect royalties or threaten the open source community.
In essence if a distributor distributes GPLv3 code to his clients/customers, he grants them a conveyable patent license. And so their modifications to the code once released will convey the patent license to every recipient.
I believe GPLv3 is a great work to come. It is still under review and comments from the public till the end of this month.
Like every programmer out there, all of us tend to commit their code in branches. As time goes, we solve bugs in those branches, and that does not mean that changes stop in the main branch (trunk). After a while we usually need to merge all those little updates done in those version specific branches, back to the main branch.
Usually, we know which files we touched and so we know which files need to be merged. Sometimes things get little bit tougher when many files have been touched, and it is hard to remember which ones.
I will try to summarize my experience with those four tools (xxdiff, kdiff3, kompare and meld) in terms of two files merging. If you want to do three way merge, then you’re only left with kdiff3. I never tried that with kompare nor meld i.e. I don’t know if they even support it.
xxdiff
That’s one of the very simple tools, with a very obvious interface. One of the advantages that I like about it is that it is very easy to use and fast to learn.
xxdiff can diff directories recursively. So if you have a CVS or Subversion tree, you can compare two sandboxes together. There is no option in xxdiff to ignore directories. That means it will compare all the CVS directories and files, a thing that I don’t like. However, there is a simple hack about this. In xxdiff options, you will find a place to write the command by which you diff files. I created a shell script and called it mydiff, which passes the first two parameters to the diff program. However, it greps all the output except any line containing CVS.
#!/bin/bash
diff -q -s -r $1 $2 | grep -v CVS
You can save those lines in ~/mydiff and inside the xxdiff options use /home//mydiff instead of the recursive diff command. A feature that makes xxdiff very simple to use is the ‘n’ button, which jumps in the files to the next mismatch.
xxdiff uses some color coding for segments included in the merge. I was able to figure out the meaning of the color codes after I did some menu selections - not the simplest way to show the functionality, but it works.
kdiff3
In my opinion, kdiff3 is the best when merging two files only and not a tree. I use it a lot when I know the pair of files that I want to merge. When trying to merge a tree, kdiff3 use color coding (red, green, yellow), which is not intuitively understood unless you read the docs. I consider it the best when merging two files, but second xxdiff and meld if merging directories. xxdiff and meld will be better choices if you want to merge directories or two sandboxes of two branches.
kdiff3 replaces the two panes in xxdiff & meld with two column in a table having icons. Some may find that better than the two panes, but I personally was not comfortable working with those icons instead of the two panes.
It has the advantage of meld and xxdiff of being able to merge over the network or more specifically using protocols handled by kioslaves components of kde. Examples include merging files (or trees) on two different computers from a third.
I face this when I’m merging files from my sandbox lying in my Linux box, which is allowed to run specific versions of software with someone’s else’s code on his Linux box, using my colinux on my windows machine, which I run all the software with the most recent versions on :), and so it runs the latest kdiff3. The merge is usually over ssh or using the fish:// protocol in KDE.
kompare
kompare has a very nice interface. It is actually the most entertaining to work with. The way it draws the comparison between diffs and shows where lines should be inserted from one file to the other is the best of all of them. If you’re merging two files and you know which files you are merging, it is good to use.
When merging trees, kompare shows only the mismatches in a very small rectangular area. It actually draws a tree inside a scroll box, which is very tiny to work in. The fact that it only shows mismatches and does not dedicate a whole screen for file mismatches made it harder for me to use when merging two cvs sandboxes.
The same network merging pluses in kdiff3 are also in kompare due to being a KDE application and thus using kioslaves.
meld
This is currently my most frequently used merging tool. It is more mouse friendly and less keyboard friendly. In comparison with xxdiff, it is very similar even the interface except it is hard to guess the keys. If you use the mouse it will be a very easily used merging tool. It draws sections and shows positions of merging in a similar way to kompare. Merging or copying portions of code is accomplished by clicking two little arrows in front of every section.
It also compares directories and shows them in two panes. The directory listings are drawn in GTK trees with arrows to open every branch. It responds to right clicks of mouse, which shows a pop-up menu to copy directories when comparing directories. When comparing CVS or SVN sandboxes meld shows buttons to ignore directories used to manage the sandbox.
Although I use meld frequently to merge CVS sandboxes, it cannot be used to merge trees or sandboxes over sftp or ssh. Unlike kompare and kdiff3 that use the kioslaves of kde to be able to open and read files and directories over various network protocols.
(Allegedly) Microsoft was behind the SCO / Linux suitcases. Now Microsoft is very upfront in their attack to Linux. Because the attack involves patents, Microsoft is focusing on the US - not in Europe, which in my opinion they won’t dare to do it there.
If Microsoft wins its battle the outcome will be depriving the US businesses from the benefits of free software - nothing will happen in the EU because the battle if there is one, will end up in US courts, and so the result is not enforceable in the EU.
I wrote a small program from a couple of days that colors the output of make. I found that it was very useful, and actually makes the output more readable and easier to spot errors in long scrolling make outputs. The program is called mkolors, short for make with colors.
mkolors is a shell script that filters the output of make using mkcolors, another C program present with mkolors. The filtered output colors parts of lines according to its understanding of what the line is.
