Yet Another Code Monkey

February 26, 2010

Progress!

Filed under: Rant — codemonkey @ 10:40 am

The bathroom here at work has always had the paper towel dispensers with the proximity sensor.  You know, the ones where you put your hand close to it and it automatically rolls out a pre-determined amount of towel for you, so you don’t have to touch the germ-filled handle the last person dirtied up.  Not too long after I started, they put in soap dispensers that work similarly.

Today, the batteries were dead in both towel dispensers and one of the soap machines.  Now, all we need is proximity-sensored faucets and we’ll be unable to perform the simple act of washing our hands without electricity.

February 16, 2010

Picture resizing

Filed under: Wordpress — Tags: , , — codemonkey @ 2:24 pm

One of the recent features of WordPress is supposed to be built-in support for image resizing, so you don’t have to do it with an external program.  Well, it wasn’t working for me.  The best I could to was insert a picture at full-size, then edit its properties while still editing the post and force a particular size.  And that meant that it would just tell the browser to display the image at a smaller size; the full-size image would still have to get sent across the wire.  In the dialog for “Add an Image”, the radio buttons for “Thumbnail”, “Medium” and “Large” were there, but not selectable.

After reading through the code, adding a zillion print statements to wp-admin/includes/media.php and wp-includes/media.php (does PHP even have a usable debugger?), searching for other blogs/forums/people having the same problem, I finally found this article where it mentions that WordPress uses PHP’s built-in image manipulation functions, which uses the GD library. Finally, a clue!

Ubuntu doesn’t install the php5-gd package by default, so a quick ‘apt-get install’, and it’s there.  Now, for any new pictures I upload, it seems to work!

Thumbnail and Medium size… Automagically!  It’d have been nice if something would have mentioned that GD was required or complained that it wasn’t installed.

Project Cleanup part 3 – More grid beam

Filed under: DIY — Tags: — codemonkey @ 10:36 am

In the last post, I made up all the beams for the kids’ new toy bins.  Because of the way these grid beam pieces connect to each other, putting it all together was pretty straightforward.

Instead of the fancy hardware the other grid beam sites mention using, I’m using regular 3-inch long 1/4-inch round-head bolts and T-nuts which get hammered into the back side of wherever the bolt is going through.

Here’s a view of the first corner joint

Corner attached

And the same joint from the other side

Corner from the other side

Here’s a picture when I had most of the frame finished with a couple of the rails that will be holding the bins.  It’s looking down the length of the frame, though the center, and it shows just how badly twisted and bent the wood is.

Twisted wood

And after wrestling to get everything to line up… The final product!

Done view 1

From the front

Done view 2

3/4 View

With the bins on:

Done view 3

Done view 4

To keep the bins from sliding off the front, I have two screws through the front rail for each bin, with a nut snugged right up to the head so it’ll stick up a bit.

February 11, 2010

Project Cleanup part 2 – Grid beam

Filed under: DIY — Tags: — codemonkey @ 5:43 pm

The kids toys downstairs have gotten completely out of hand.  I’m going to make a stand to hold some bins that we can put some of the toys into.  You know what I’m taking about… wooden sides with plastic bins tilted a bit so you can reach into them.  I’ve seem some for sale at the store, but they seemed pretty expensive for what you get.

A few months ago, I saw a blurb about Grid Beam on Hack-a-Day.  There are a few other useful links out there (Makezine and gridbeam.biz) and I made a mental note to look into it some more the next time we needed any kind of furniture, especially for the kids.  So, I sketched out a rough diagram of what it would look like.  3 bins high and 3 wide.  I already had 5 bins; I needed 4 more plus the wood and hardware.  It would end up about 4 feet high, 8 feet wide and 16 inches deep.

Grid beam is a building system where you use sticks (wood or metal) with regular-spaced holes the same distance apart as your sticks are wide.  The pieces are fastened together with bolts.  Because of the hole spacing, the corners come together in a way that help make the whole thing square.  I used 2x2s, which are actually 1 1/2 inches wide, so I’d need holes every inch and a half.

Most of the grid beam pages I looked at recommended using special bolts and nuts called Joint Connector bolts, like these (link type BA or BB).  With these, the heads are flush with the wood so you won’t hurt yourself if you bump into it, which is important since the kids will be using it.  I couldn’t find these locally, and they seemed pretty expensive to order, so I ended up using 3-inch long 1/4-inch bolts and T-nuts.  The nuts required the final hole size to be 5/16-inch.

The list of supplies:

  • 12 8-foot 2x2s.  I only ended up needing 8, but now I have extras
  • 1 3-foot pieces of 2-inch wide aluminum angle stock for the jig
  • 3 5/16-inch drill bits.  I’m drilling hundreds of holes, so they’ll be wearing out.
  • a box of 3-inck long 1/4-inch bolts.  Round head that takes a phillips driver.
  • a box of T-nuts the bolts fit into.

The first task was building a jig for drilling all those holes.  I used a 3-foot piece of aluminum angle stock.  Clamp the ruler to the aluminum at both ends and mark the metal every inch and a half.

Mark hole locations

Now, I’ve already made two mistakes.  The first one is that line all the holes are on should be a distance of 1/2 of the hole spacing from the corner of the jig.  Here, I’ve clamped the ruler 1 1/2 inches (one whole hole spacing) where it should have been 3/4 inch.  The second problem is that the first hole should also be a half-hole-space from the end, not a whole-hole-space.

Anyway… score a line along the length of the ruler.

Scored along the ruler

And drill a pilot hole each place where the lines cross

Pilot holes

Here’s a picture of the jig when all the holes are drilled at the correct 3/4-inch spacing from the angle: the holes on the top.  The holes on the bottom edge are the incorrect 1 1/2-inch distance.  I guess I could use it for 2x4s…

Final holes

And finally, after cutting off 3/4 inch from the end

Mistake

At this point, I have a good jig,  The next issue is that when I went to the lumberyard to get 2x2s, it was pretty slim pickings.  Pretty much every single piece was bent and/or twisted, so I could only pick out the least warped pieces.  Next time, it might make more sense to get 2x4s and rip them down to 2x2s myself, since 2x4s are less likely to be really warped.

I don’t have a drill press either, so the best plan was to take several clamps and force the wood straight, at least long enough to drill the holes.  Oh yeah, and hope the holes are straight.

clamped wood

Since my wood is 8 feet long and the jig is only 3 feet, after drilling a batch of holes, I had to move the jig down and reclamp it.  I used a 5/16 bolt to position the jig in the last hole drilled before I moved the jig.

More holes

A dozen 8-foot 2x2s and about a zillion holes later, I’m done drilling.  The jig is pretty much worn out from the side of the drill bit eating away at the holes.  If I’d used steel for the jig, or had a drill press to make cleaner holes, it might not have been so bad.  The picture isn’t really clear, but it’s supposed to show the 5/16 drill bit (the original size) in one of the holes.  I was able to fit a piece of 14-gauge wire in the space, but not 12-gauge, which means it’s 0.064-inch (just over 1/16) oversized.

Worn out jig

Worn out jig

Finally, here’s a picture of the T-nuts I’m using, in case you’re not familiar with them

T-nut

February 9, 2010

Project Cleanup

Filed under: General — Tags: , — codemonkey @ 2:21 pm

I’ve taken a couple of weeks off from work to spend at home cleaning and organizing the house.  Believe me, it needs it.  No, I’m not taking picture of the mess; it’s that bad.  Yes, two weeks isn’t too long to take off.  Unfortunately, the first week was used up taking care of sick kids.  *shrug*  It happens.  And, it’s hard to get much done with both kids running around – they weren’t that sick.

So, anyway, I have a few old Discover magazines, and some computer catalogs I’ve kept since high school for reasons unknown.  It’s interesting to page through them, especially for the advertisements.  I’ve scanned a few of them for posterity so I can recycle the paper magazines.

This batch is from the November 1986 issue.  First up is a short article about the contents of Coca-Cola, which had just switched from the “original” formula to New Coke and back again about a year earlier.

Is any Coke the real thing?

Before PCs with word processors were widespread, electric typewriters got the ability to edit documents before typing them out.

Xerox 6020 Typewriter

I thought a Honda Civic was pretty much the definition of a cookie cutter car…

Honda Civic page 1Honda Civic page 2

Back when Back when AT&T (and Bell Labs) actually invented things

ATT page 1ATT page 2

Part of an 8-page centerfold advertisement

Apple page 1

The next two pages show a classroom with the title text “Your Kids.” The picture shows 6 or 7 kids each using some educational software on different Apple computers. Then, on the inside:

Apple page 3Apple page 4

Apple page 3Apple page 4

This poster looks pretty cool, though I certainly have nowhere to put such a thing

History poster

February 2, 2010

Critical Section Locking

Filed under: Software — Tags: , , — codemonkey @ 2:51 pm

Coming up with an IPC locking mechanism that is robust and works between more than one computer is a hard problem.  Our needs for locking at work have been pretty light up until recently.  We’ve relied on our ORM grabbing object IDs from sequence in the database for the most part to avoid needing to worry about locking, or at least push it down to being the database’s problem.  Still, some code sections need locks around them.

The first implementation of a multi-machine locking mechanism involved a simple database table with a unique constraint to store the resource ID that you’re locking.  It works pretty well, you just have to remember to do all your lock-related transactions on a separate database handle so it can be committed independently of other transactions you may have going on, and it doesn’t support shared locks unless you look into stored procedures or some other nonsense.  Our database is also very busy, and one slow or hung process can block access to that lock table for everyone else.  Pretty soon, the database is being hammered by queries on that one table.

The next pass was to use the existence of a directory on a shared NFS filesystem to represent the lock; mkdir(2) is atomic, even across NFS.  Then we started dropping a file in the directory with information about who created it to support cleaning up the lock being held by a hung or crashed program.

That introduced a bug because of the interaction with the NFS servers.  Say process A has the lock.  Process B wants to check on the staleness of the lock, and so opens the info file and starts to read it.  Process A now wants to give up the lock, and so deletes its info file and tries to remove its lock directory, which actually fails.  For our NFS systems, if process B has the file open and process A unlinks the file, the NFS server creates a file called something like .nfslock1234blah in the same directory to keep track of it.  Since the directory is not empty, process A can’t remove the lock directory and the lock is now hung.  We’ve tried different wait times, dancing directory names before deleting them, nothing could reliably get rid of the issues.

Oh yeah, this mechanism doesn’t support shared locks, either.

Our requirements have changed, and now we need shared locks.  We’d also like to continue to use the NFS filesystem to communicate about the locks, because it’s well maintained by our sysadmins, available on all the machines in the cluster, and its behavior is well known.  Another possibility was to create a locking daemon running somewhere on the cluster, but then the sysadmins would have to support it and make sure it was always running and saving state reliably during downtime.

The first thing we can do is to make the locking mechanism to a two-step process.  A locker first declares its intention to lock by dropping a uniquely named subdirectory in the lock directory as a reservation.  To actually acquire the lock, it must create a symlink with a well-known name in the lock directory pointing to its reservation; symlink(2) should also be atomic.   We’ll compose these reservation names by using the hostname, processID and the current time.  To get the lock, the process tries to symlink() to their private reservation directory.   To release the lock, they first remove the symlink (someone else can now claim it) and then remove the reservation directory.

Contents of the lock directory

First lock directory scheme

We can also easily add in shared locks by having a common directory where all the shared lockers drop in a uniquely named file or subdirectory.  Locking is slightly different in that you get the lock if the symlink() succeeds, or if the symlink already exists and it points to the shared directory.  Unlocking a shared lock requires that you first try to remove the shared directory, and only remove the symlink if the rmdir() succeeded.  This way, if other shared locks have files in the shared directory, the rmdir will fail, but that process can still give up it’s hold on the lock while keeping the shared lock alive.

There’s still a race condition there…Process Y has the only shared lock and wants to give it up.  It rmdir()s the shared directory, which succeeds.  Meanwhile, process Z wants the shared lock.  It mkdir()s the shared directory, sees that the symlink points to the shared directory, and assumes it has the lock.  Now, process Y continues by removing the symlink.  Process Z now thinks it has the lock, through someone else can also claim it.

To fix that, shared lockers creating a new shared directory use a name that matches a pattern so others can find it, but is still unique when it gets created.  The next process that wants the shared lock will first look to see if any shared directory exists and drops a file in there.  In the race condition above, Z will create a new directory but readlink() will point to some other directory name, and so it knows it does not yet have the lock.

Final lock directory scheme

Final lock directory scheme

Here, PID 554 on host 2 is in the process of giving up the lock. The reservation file and directory have been removed, but the lock symlink still points to the deleted shared-lock directory.  Meanwhile, two other processes are requesting a shared lock and three others are waiting on an exclusive lock.

Probably the most important thing to remember about avoiding race conditions is don’t check for some condition and then alter the filesystem based on the return value.  Instead, try and do the change and then check the return value and allowed failure modes.  For example, don’t check for directory’s existence and then create it if it doesn’t exist.  Another process can sneak in between the stat() and mkdir().  Instead, just do the mkdir() and know that it can fail with EEXIST if the directory was already there, and another error like EPERM would be a fatal exception.

This should cover all the bases.  Be thorough about checking return codes from syscalls, add in some stale lock cleanup code, zillions of test cases and we should be good to go.

January 18, 2010

The best feeling

Filed under: Kids — Tags: , , — codemonkey @ 7:53 pm

15 or 20 minutes in a rocking chair with a sleeping baby on your chest has got to be about the best feeling in the world.

January 8, 2010

iTerm for the win

Filed under: Software — Tags: , , , — codemonkey @ 10:58 am

I’ve just recently started doing some serious development on my Mac.  For now it’s command-line stuff with Perl, and using vim for editing.  Terminal.app that comes with OSX has been pissing me off pretty much every day.  I’m used to doing most development on a Linux box where the Gnome and KDE terminal emulators have many behaviors that I’ve become used to.

After the third or fourth time hearing me complain about it, a friend reminded me to give iTerm a try.  Holy cow, it fixes pretty much every complaint about Terminal.app

  • Put a border at the bottom of the window so stacked terminals don’t blend together
  • Home key takes you to the beginning of the line instead of the beginning of the history
  • control-pageup and -pagedown move back and forth in the history
  • copy-and-paste work like an xterm
  • double-clicking on a word highlights the whole word, even if it has dashes or slashes in it
  • has a setting for focus follows mouse

The only thing I can think of to add is have a setting to autoraise the active window.  Maybe I can look into that this week

Circular constraints

Filed under: Perl — Tags: , — codemonkey @ 9:35 am

I writing a little program to track inventory.  As you’d expect, there’s a table for orders (with an ID, order name and date), and another table for an order’s items (ID, item_id, order_id, count, type).  Previously it tracked completed sales transactions, purchases and physical inventories.

I’m adding a new feature where it will now also track customer’s orders, but before the sale has been completed.  I’m calling them PickLists because I use that info to generate a pick list you can use when collecting all the order’s items.  It’ll make a PickList record and all the order items will point to that order.  Then, when you’re completing the sale, it will create a Sales order and start pointing the items to the sales order instead of the PickList order as you scan them out.  This way it can track that you’re actually packing all the right items that the customer ordered.  At the end, all the items point to the Sales order and we can delete the PickList order.  Simple, right?

Except that the orders.order_name column has a unique constraint.  When the program goes to save all the changes, there’s a problem because of a circular dependancy:

  1. You can’t insert the Sales order row because there’s a PickList order with the same order_name.
  2. You can’t remove the PickList order because the order items are still pointing to it
  3. You can’t point the order items to the Sales order because it hasn’t been inserted yet.

We have some code in testing for our ORM to detect this case with a circular dependent chain of foreign key constraints.  If you allow that column to be nullable, then it’ll insert all the new rows with NULL in that column, then go back and update the column with proper IDs.  My first thought was to adapt that code to handle this new case.

But from a data consistency standpoint, it seemed less wrong to change the order table’s unique constraint to cover (order_name, type) instead of allowing an order item to belong to no order.

January 6, 2010

More disk problems!?

Filed under: Computers — Tags: , , — codemonkey @ 9:26 am

Last night I had another scare about failing disks.  Like I mentioned earlier, the new fileserver machine has a RAID1 pair of disks for the system drive, one partition each for the root filesystem, and another for swap.  That way, the system can keep going even if there’s a bad sector anywhere on either disk.

I got an email that said both RAID sets were running in degraded mode, and I got that sinking feeling again.  The same one I had after I’d lost all that data earlier.

Code:
# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sda1[2](F) sdb1[0]
3903680 blocks [2/1] [U_]
bitmap: 4/239 pages [16KB], 8KB chunk

Code:
md0 : active raid1 sda2[2](F) sdb2[0]
308664768 blocks [2/1] [U_]

The good thing was that everything was still running.  The logs were strange in that both the root and swap md devices had failures at the same time, meaning it was probably a problem with the drive itself, and not bad sectors on the disk.  Also, the /dev/sda device node was missing completely, and the syslog had the message “ata5: SATA link down”.  Maybe it was just a cable problem *cross fingers*

/dev/sdb was still up, so I could use smartctl -a to find the serial number of the working disk – the other one must be the problem drive.  I laid my hand on the non-responsive one; it was cool and not vibrating.  I jiggled the cable and the disk started spinning up!  The syslog showed the disk was recognized and active.

The minor issue was that I now couldn’t remove the failed disk from the RAID, since the device manager had removed the /dev/sda device node when the cable came loose:

Code:
# mdadm /dev/md1 -r /dev/sda1
mdadm: cannot find /dev/sda1: No such file or directory

No problem, just add the partition under the new name

Code:
# mdadm /dev/md1 -a /dev/sdd1
mdadm: re-added /dev/sdd1
# mdadm /dev/md0 -a /dev/sdd2
mdadm: re-added /dev/sdd2
# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdd1[1] sda1[2](F) sdb1[0]
3903680 blocks [2/2] [UU]
bitmap: 0/239 pages [0KB], 8KB chunk

Code:
md0 : active raid1 sdd2[2] sda2[3](F) sdb2[0]
308664768 blocks [2/1] [U_]
[>....................]  recovery =  0.0% (95488/308664768) finish=107.7min speed=47744K/sec

<code>unused devices: <none>

So, /proc/mdstat still lists the failed drive, but the RAID sets are now working normally, and the failed device should disappear the next time it gets rebooted.

« Newer PostsOlder Posts »

Powered by WordPress