rsync -uwill ignore any files modified later on the target than on the source. This is good, but I wasted a bit of time today trying to figure out why
-cwould not pick up files with different MD5 sums. The conclusion:
-uwas filtering them out before getting to the MD5 check.
- Quite often I use something like:
cat file | while read line; do
If you have any program contained within the loop – ssh in my case today – which opens standard input, for any reason, it will drain the contents of the file that’s being cat’ed, so you will only go around the loop once. To avoid this, put
< /dev/nullat the end of the
sshcommand, it will prevent any more of this nonsense (which is actually very sensible)!
Having chipped away for a few hours over the weekend, I’ve cobbled together this nice little script that can be used standalone (from the command line), or as a module in your own python applications. The script basically measures the amount of data being transferred and the time taken to transfer it, this info is spewed out along with a calculation of the average throughput.
Command Line Usage
tar cz /home/ | speedometer.py > archive.tar.gz
Use as a Python Module
from speedometer import Speedometer
speedo = Speedometer(sys.stdin, file('/large_file.iso'), sys.stdout)
tar cz /home/ | speedometer.py > archive.tar.gz
Use as a Python Module
import sys from speedometer import Speedometer speedo = Speedometer(sys.stdin, file('/large_file.iso'), sys.stdout) speedo.start()
The Wikipedia article on ANSI Escape Codes was referenced in the making of Speedometer.
- The directory that’s created using
django-admin startproject <xxx>. A project is a collection of configuration and apps for a particular website.
- One Django installation can be used for multiple websites. The Site framework gives applications the ability to base logic on this fact.
- Apps provide specific features for the Django site, for example a blog app, a poll app, etc. An app can be in multiple projects. A project can contain multiple apps.
- A model generally corresponds to a database table. Models are objects in the sense of object-oriented programming and have methods to manipulate the object or make it do all kinds of processing.
- This is the HTML (or other) presentation layer.
- This is the logic layer. urls.py associates Views and Templates.
- Each Django project is equivalent to a Python package. Packages are directories containing a collection of modules (files with classes, functions and variables).
These are the initial steps to get up and running with Django. First you will have to install it – instructions vary based on your OS, but check out the Django Quick Install Guide if you need help.
django-admin startproject <web_site_name> cd <web_site_name> edit settings.py ./manage.py syncdb ./manage.py startapp <app_name> edit settings.py # to include <app_name> in INSTALLED_APPS edit <app_name>/models.py ./manage.py syncdb edit <app_name>/admin.py # to allow Web Admin GUI access to models (DB tables)
So, yes, I have put these instructions here; they were primarily as an aide-mémoire, but I would strongly recommend following the superb documentation on the Django website.
These are general Python best practices, not solely related to Django.
Source Encoding Definition
As recommended by PEP-0263, a coding ‘magic comment’ should be included in all Python source files on line 1 or 2, for example:
#!/bin/python # -*- coding: utf-8 -*-
Single line docstring
""" This is a single line docstring. """
Note: You can prepend ‘r’ if there are backslashes in it, and prepend ‘u’ if it’s a unicode string.
""" This is a multiline docstring """
""" This is a multiline docstring """
Don’t Repeat Yourself (DRY)
This is a philosophy that is sometimes not as easy to follow as you might initially think. It is a great habit to get into though since bugfixes and maintenance efforts can be concentrated on specific parts of the code.
Last Christmas, I was the delighted receiver of an Epson EB-S02 Projector. It’s fairly straightforward, but has a couple of nice features:
- Display via USB
- Serial (USB) Control
Moving on… It turns out that the projector has just about the perfect throw angle to completely cover one wall in our office when it’s placed in a cupboard at the opposite side of the room – Nice… That inspired me to do a miniature overhaul of the office to facilitate games, films and work.
The overhaul will consist of the following:
- Ceiling mount the projector
- Electrical outlet near to the projector
- Removable light fitting for the main lamp in the room
- VGA extension from PC to Projector
- New screen
- Bespoke desk
Ceiling mount the projector
I’ve always wanted to do a ceiling mount. Any time I’ve had the loan of a projector, I’ve found it annoying to have to sit low in my seat, etc. just to avoid casting a shadow on the screen. I’ve always had to construct flimsy stands to hold the projectors which made me nervous every time.
As I mentioned above, the projector’s throw angle was just right to fill an entire wall of our office, but only if the projector was in the wardrobe at the opposite side. Fine. This ceiling mount was not going to be on the actual ceiling, but at the top of the wardrobe instead.
Rather than fork out > €100 for the official Epson projector ceiling mount, I decided to do a cheapo job on it and put my imagination and DIY skills to the test. First challenge, design the mount.
Designing the Mount
Sketching ideas out really helps to figure out how things will fit together, how things will work, problems with the design and potential trouble while building the design. I went through three iterations before I came up with my final design.
Remembering the packing materials I used to use under projectors in the past, so that the image would hit the screen perfectly, I started out sketching a mount that would have flexibility of positioning built in.
This quickly got out of hand, and became something I would fear building rather than enjoy. What’s the point in that.
Simplifying the design, I came to the conclusion that one flat plate would suffice to hold the projector – no need for a metal bender. It would be secure provided there was enough of a ledge to hold the plate in place.
In the final round of simplification, I realised there was no need to do any fancy cutting on the wood that I had to make a ‘ledge’, I could just cut them to length and screw them up to the sides of the wardrobe. The plate would sit just on top. If any tilting of the projector had to be done, I would use small wedges / packing material – this installation is permanent, so I can set and forget!
From a previous projector related project, I had a couple of 2″x2″ (40 mm x 40 mm) lengths of wood that would do nicely for the runners.
I had no metal, didn’t really know where to go to get some, and didn’t really want to pay for any. I liked the idea of salvaging some material for this project, so I thought long and hard about where I might get some metal.
I went for a walk one day to have a look in some skips. There were no obvious candidates ripe for recycling. While walking through a homewares shop, I saw baking trays; they looked just about perfect. I hadn’t got any measurements with me, nor a measuring tape. I kept the baking trays in my mind as a fallback plan in case I couldn’t fulfill my desire to salvage.
Finally it hit me. The shell of a PC would be about the right size and strength to hold the projector. I investigated where our local electrical waste depots were, and paid a visit a day or so later. After a small bit of evasion and a white lie or two to prevent my forced ejection from the depot, I found exactly what I was looking for. A tower PC case, the perfect size that came apart in three sections: each side, and the top. I was pretty sure I’d only need one side, but what the hell, I took the lot with potential future changes to my plan in mind.
I purchased these M4 x 25 mm bolts from CPC which were probably above spec, since I couldn’t find any alternatives in the local stores I checked out. These would be used to fix the projector to the metal plate.
A few screws for the runners and a bit of paint – I had all the materials I needed.
To achieve maximum support in as many positions as possible, I decided to have the runners go the full depth of the cupboard. This would allow the projector to be slid to the front, back or anywhere between in the hunt for max. image splash on the far wall.
With the old carpentry proverb ‘measure twice, cut once’ in mind, I went about marking up the wood and getting a tennon saw to chop the wood to length.
The screws I happened to have were not quite long enough to make it all the way through the runners and into the side of the wardrobe, so I drilled out four countersinks in each runner to get the grip they would need.
It would have been a good idea to drill some pilot holes in the side of the wardrobe to help get instant grip when the screws passed through the runners. Not thinking of this, I just held the runners as tight against the walls to help force the screw through and keep the runner and the wall mated tightly.
Once the runners were in position, I masked around them and stirred up the white gloss paint. Four coats later and educated about how thirsty end-grain can be, this part of the job was done.
The manual for the projector has a mounting diagram which showed the layout and measurements for the centres on the underside of the projector. I marked these out on the metal plate and wasn’t too concerned about accuracy – the plan was to turn the holes into slots to further enhance the adjustability of this DIY bracket.
I used a hammer and nail to punch homing guides for the drill in the metal and then got to work drilling. After 10 minutes at the helm, I realised that this was going to be a battle. My hand and arm were sore, the metal was steaming and the chair on which I was drilling had scorch marks. Oil, water and adequate rest breaks were used to keep things under control, and another 20 minutes or so later, I had all three holes drilled.
Moments later I had a sacrificial bolt hammered through the holes and pulled in every direction to help loosen the grip of the plate on the final bolts that would be there. The tearout on the other side of the metal left a lot to be desired – I took a file to it, but really it’s still not great.
I affixed the projector to the plate, loosely on all bolts at first, and then tightened all together (if I did two bolts all the way, the last one wouldn’t tighten). And finally slid the plate onto the runners. It all fit together perfectly!
And, elated with the success, I gave way to the inner geek and loaded the projector’s own test grid first. Ahh! Nice! It did take some time and a few cardboard shims to get it this good, and even still there’s room for improvement (although, I’m tending to think the walls in the room aren’t parallel or perpendicular to the floor and ceiling – who knows).
Now, this has literally taken me months to get from draft to published, so I’m not promising anything about part 2. If you’ve been paying close attention, you will see that the electrical outlet is installed, I just have to write down a few words…
There is a decent article on the differences between the two configurations on RedHat’s website: https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Cluster_Administration/ap-ha-halvm-CA.html
That article boils down to this:
Use HA-LVM if the application accessing the storage runs on one node at a time. This scheme uses LVM host filtering and tagging on either the VG or LV to ensure that storage is accessed by only one host for any period of time.
Use CLVM if the application accessing the storage is cluster aware and can be run on multiple nodes at once. CLVM uses clustered locking services to prohibit access to shared storage while logical volumes are being configured.
But it doesn’t end there. Within HA-LVM, there are two methods for ensuring logical volumes are activated on one node at any time: RedHat recommend using CLVM locking even for exclusive access to the shared LVs (it’s simpler to setup and less error prone). The other method employs LVM tagging to mark a volume group or logical volume as in-use by a particular node.
CLVM and HA-LVM with CLVM locking require the High Availability Add-On and Resilient Storage Add-Ons. Key to the operation of CLVM are:
clvmd – distribute LVM metadata across the cluster.
cmirrord (only when clustered mirroring is in operation) – tracks mirror log information in a cluster.
A summary of locking types as seen in
- Locking Type 0
- No locking (dangerous: risks metadata corruption if LVM2 commands get run concurrently)
- Locking Type 1
- Local file-based locking. Also used in conjunction with HA-LVM using tagging
- Locking Type 2
- Uses an external locking library
- Locking Type 3
- clvmd for HA-LVM Failover with CLVM or CLVM on it’s own
- Locking Type 4
- Read-only locking
simply switches the locking type to 3, and a corresponding Mcode>–disable-cluster changes it to 1.
Yesterday I released a small utility on Github. IP Mon is a free and open source shell script that can be used to monitor the network status of many servers on one screen. I wrote it as a very lightweight utility that wouldn’t require any software to be installed on remote machines. IP Mon has a configuration file,
~/.ip-mon by default, that is used to group and list all servers to be monitored. I use it every day at work and find it quite useful.
I am keen to get any pull requests for improvements to the utility.
Having used kickstarts a lot to build and test new Linux systems, I had a pain in my hand retyping the same things into the installer boot prompt, for example:
linux ks=http://myserver.example.com/kickstart/sl-63-minimum-ks.cfg ip=192.168.4.5 netmask=255.255.255.0 gateway=192.168.4.1 dns=192.168.2.9 ksdevice=eth0
…and so on, depending on what’s being tested.
Inevitably, there would by typos from time to time which was bothersome.
I had a thought; why not make a bootable Netinst CD to save me having to type all of this, every time… Turns out it’s really easy.
Step 1: Download the template CD
Most of the systems I work on are Scientific Linux, so I headed to my nearest mirror site to download the (small) netinstall ISO. These instructions should work on Red Hat, CentOS and Fedora too.
The ISO we’re looking for can be found in the following location within the Scientific Linux tree on most servers:
6.4/x86_64/iso/SL-64-x86_64-2013-03-18-boot.iso, or on the installation DVD:
Step 2: Prepare a working tree
Once you’ve downloaded the ISO:
# Mount it mount -ro loop boot.iso /mnt # Make a working directory to make your custom CD mkdir customdisk # Copy the contents from your template disk cp -a /mnt/* customdisk # Modify permissions for mkisofs chmod -R +w customdisk
Step 3: Modify the boot menu
Edit the file:
customdisk/isolinux/isolinux.cfg. In the section that gives the list of boot options, copy the default one and remove the
menu default line. You should have the following then:
label linux menu label ^Install or upgrade an existing system menu default kernel vmlinuz append initrd=initrd.img label linux menu label ^Install or upgrade an existing system kernel vmlinuz append initrd=initrd.img
Next you need to give the copy unique labels, and finally add what you need to the append line. You could be left with something like this:
label linux menu label ^Install or upgrade an existing system menu default kernel vmlinuz append initrd=initrd.img label kickstart menu label My Custom Kickstart kernel vmlinuz append initrd=initrd.img ks=http://myserver.example.com/kickstart/sl-63-minimum-ks.cfg ip=192.168.4.5 netmask=255.255.255.0 gateway=192.168.4.1 dns=192.168.2.9 ksdevice=eth0
Repeat Step 3 as many times as you need.
Step 4: Prepare the ISO for burning
Thanks to the SYSLINUX Wiki article: http://www.syslinux.org/wiki/index.php/ISOLINUX, I knew to run the following command:
mkisofs -o newCDimage.iso \ -b isolinux/isolinux.bin -c isolinux/boot.cat \ -no-emul-boot -boot-load-size 4 -boot-info-table \ customdisk
Step 5: Burn the disk
I’ll leave it to you to decide the best way to burn the CD. I mostly test on virtual machines, so don’t need to burn.