a dumbed down version of wpdb for sqlite
I’ve been working, gradually, on a project using an sqlite3 database (for its convenience) and found myself missing the clean elegance of wpdb… so I implemented it. It was actually really easy to do, and I figured I would throw it up here for anyone else wishing to use it. The functionality that I build this around is obtainable here: http://php-sqlite3.sourceforge.net/pmwiki/pmwiki.php (don’t freak… its in apt…)
With this I can focus on the sql, which is different enough, and not fumble over function names and such… $db = new sqlite_wpdb($dbfile, 3); var_dump($db->get_results(”SELECT * FROM `mytable` LIMIT 5″));
the code is below… and hopefully not too mangled…
< ?php
class sqlite_wpdb {
var $version = null;
var $db = null;
var $result = null;
var $error = null;
function sqwpdb($file, $version=3) {
return $this->__construct($file, $version);
}
function __construct($file, $version=3) {
$function = "sqlite{$version}_open";
if ( !function_exists($function) )
return false;
if ( !file_exists($file) )
return false;
if ( !$this->db = @$function($file) )
return false;
$this->version = $version;
$this->fquery = "sqlite{$this->version}_query";
$this->ferror = "sqlite{$this->version}_error";
$this->farray = "sqlite{$this->version}_fetch_array";
return $this;
}
function escape($string) {
return str_replace("'", "''", $string);
}
function query($query) {
if ( $this->result = call_user_func($this->fquery, $this->db, $query) )
return $this->result;
$this->error = call_user_func($this->ferror, $this->db);
return false;
}
function array_to_object($array) {
if ( ! is_array($array) )
return $array;
$object = new stdClass();
foreach ( $array as $idx => $val ) {
$object->$idx = $val;
}
return $object;
}
function get_results($query) {
if ( !$this->query($query) )
return false;
$rval = array();
while ( $row = $this->array_to_object(call_user_func($this->farray, $this->result)) ) {
$rval[] = $row;
}
return $rval;
}
function get_row($query) {
if ( ! $results = $this->get_results($query) )
return false;
return array_shift($results);
}
function get_var($query) {
return $this->get_val($query);
}
function get_val($query) {
if ( !$row = $this->get_row($query) )
return false;
$row = get_object_vars($row);
if ( !count($row) )
return false;
return array_shift($row);
}
function get_col($query) {
if ( !$results = $this->get_results($query) )
return false;
$column = array();
foreach ( $results as $row ) {
$row = get_object_vars($row);
if ( !count($row) )
continue;
$column[] = array_shift($row);
}
return $column;
}
}
?>
Subscribe to the comments for this postPostfix, DKIMproxy, Spamc
If you’re running any moderately busy mail server you’re probably using spamassassins spamc/spamd to check for spam because its tons more efficient than piping the mail through the spamassassin cli. Assuming that you do, and that you plan on adding DKIM proxy to the mix to verify and sign emails you need to put things in the right order, to save you some headache here’s what I did:
- smtp|smtps => -o smtpd_proxy_filter=127.0.0.1:10035 # outgoing dkim verify port
- 127.0.0.1:10036 => -o content_filter=spamassassin
- spamassassin => pipe user=nobody argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient} # this delivers to the “pickup” service
- pickup => -o content_filter=dksign:127.0.0.1:10037 # outgoing dkim signing port
- 127.0.0.1:10038 => -o content_filter= # the buck stops here
If you arent careful with these (which I wasnt) you’ll end up causing an infinite loop between your filters (which I did). Thus concludes our public service announcement.
Piracy and Copy Protection
There’s been a lot of hooting and hollering in the media about copy protection and piracy. DRM seems to be a rallying cry for the younger generation. Which makes sense. I figured I’d throw my own $.02 out there to the wind (for all the good its worth) But I’d first like to say that (as is true with all generalizations) this doesnt apply to all people all the time.
You really have 3 classes of people who might buy a game. You have the people with enough disposable income not to care, they just buy their game, and use it as directed. (please not I’m not discussing the as directed part, simply the who’s yelling part. I fully agree that as directed should come outside the bos — or be returnable — and should also be reduced to lay speak since most of us dont speak asshole lawyer.) Those people really arent who’se yelling about DRM.
You have the gaming middle class who, more or less, wants to go straight. They buy what they can when they can, but at $60 a pop they cant afford *everything* they want. To varying degrees they have a couple of options. They can pick what they *think* they’ll like best, and abstain from everything they cant afford, or they can “test-run” the game and see whether they like it first. As someone who has bought a car before I know which one of those options makes the most sense to me… This stage is definitely not a plateau, though, it slopes down from the previous upper class to the next… lower class.
Finally there’s the poor people. They have a computer.. maybe handed down.. maybe cannibalized… Maybe they’re preteen to mid/late twenties. Maybe they’ve got a job, maybe they dont. Maybe they’re in school, maybe not. But the common denominator here is that you arent going to see a red penny from these people no matter what you do. One does not squeeze blood from a stone. These people have WAY more time than they have money, and so spending 2 days downloading, and 2 days searching for an install key, then a crack, is NOT a problem for them… It’s cheaper than buying the thing. What, I think, game companies are missing is that a good number of these people WANT to be in the middle and upper gamer class, and when they get there they *will* spend money, because they’ll have more of that then time.
What I think *IS* interesting is the time to money ratio. You develop a game that might take 10-30 hours to beat (in the case of non mmorpgs) or last forever (in the case of mmorpgs.) Essentially you’re making something that appeals TO the people who have more time than money… Much like payday loans your target audience is pretty dangerous in general, fiscally. And as people progress to upper gamer class you have way more money than time. And you wonder why people are willing to fork over $30 to get a bunch of WoW gold so they can kill something besides “soft fluffy bunnies, which pose no threat to humanity” without having to neglect their wife and kids for 400 straight hours.
Crucify the young for being young, crucify the old for being old. It makes you wonder… in the minds eye of the video gaming industry… does their ideal customer base actually exist? I bet it lies dangerously, and precariously, in the same kind of balance that the tobacco industry lies: hook em while they’re young and bleed em dry as they age. Except the tobacco company does it better, they dont actively vilify their young “patrons” they simply wait.
Writing your own shell in php
I’ve always wanted to write my own simple shell in php. Call me a glutin for punishment, but it seems like something that a lot of people could use to be able to do… If your web app had a command line interface for various things… like looking up stats, or users, or suspending naughty accounts, or whatever…. wouldnt that be cool and useful? Talk about geek porn. Anyways this this morning I got around to tinkering with the idea, and here is what i came up with… It’s rough, and empty, but its REALLY easy to extend and plug into any php application.
apokalyptik:~/phpshell$ ./shell.php /home/apokalyptik/phpshell > hello hi there /home/apokalyptik/phpshell > hello world hi there world /home/apokalyptik/phpshell > cd .. /home/apokalyptik/ > cd phpshell /home/apokalyptik/phpshell > ls shell.php /home/apokalyptik//phpshell > exit apokalyptik:~/phpshell$ ./shell.php
See the source here: shell.phps
Internally Caching Longer Than Externally Caching
We use varnish for a lot of our file caching needs, and recently we figured out how to do something rather important through a combination of technologies. Imagine you have backend servers generating dynamic content based on user input. So your users do something that fits the following categories:
- is expensive to generate dynamically, and should be served from cache
- many requests come in for the same objects, bandwidth should be conserved
- doesnt change very often
- once changed needs to take effect quickly
Now wish varnish we’ve been using the Expires header for a long time with great success, but for this we were having no luck. If we set the expires header to 3 weeks, then clients also cache the content for 3 weeks (violating requirement #3.) We can kill the Expires header in varnish at vcl_deliver, but then clients don’t cache at all (#2.) We can add Content-Control, overwrite the Age (otherwise reported Age: will be greater than max-age), and kill the Expires headers in the same place, but this isn’t pretty, and seems like a cheap hack. Ideally we could rewrite the Expires header in varnish, but that doesn’t seem doable.
So what we ended up doing, was header rewriting at the load balancer (nginx.) inside our location tag we added the following:
proxy_hide_header Age; proxy_hide_header Expires; proxy_hide_header Cache-Control; add_header Source-Age $upstream_http_Age; expires 300s;
Now nginx setsa proper Cache-Control: and Expires: headers for us, disregarding what varnish serves out. Web clients dont check back for 5 minutes (reusing the old object) and varnish can cache until judgment dat because we get wild card invalidation
Isn’t technology fun?!
Command line arguments in bash scripts
This is something that has always annoyed me about bash scripts… The fact that it’s difficult to run
/path/to/script.sh --foo=bar -v -n 10 blah -one='last arg'
So I decided to write up a bash function that let me easily (once the function was complete) access this type of information. And because I like sharing, here it is:
#!/bin/bash
function getopt() {
var=""
wantarg=0
for (( i=1; i< =$#; i+=1 )); do
lastvar=$var
var=${!i}
if [ "$var" = "" ]; then
continue
fi
echo \ $var | grep -q -- '='
if [ $? -eq 0 ]; then
## -*param=value
var=$(echo \ $var | sed -r s/'^[ ]*-*'/''/)
myvar=${var%=*}
myval=${var#*=}
eval "${myvar}"="'$myval'"
else
echo \ $var | grep -E -q -- '^[ ]*-'
if [ $? -eq 0 ]; then
# -*param$
var=$(echo \ $var | sed -r s/'^[ ]*-*'/''/)
eval "${var}"=1
wantarg=1
else
echo \ $var | grep -E -- '^[ ]*-'
if [ $? -eq 0 ]; then
# the current one has a dash, so cannot be
# the argument to the last parameter
wantarg=0
fi
if [ $wantarg -eq 1 ]; then
# parameter argument
val=$var
var=$lastvar
eval "${var}"="'${val}'"
wantarg=0
else
# parameter
if [ "${!var}" = "" ]; then
eval "${var}"=1
fi
wantarg=0
fi
fi
fi
done
}
OIFS=$IFS; IFS=$(echo -e "\n"); getopt $@; IFS=$OIFS
now at this point (assuming the above command line parameter and script) I should have access to the following variables: $foo (”bar”) $v (1) $n (10) $blah (1) $one (”last arg”), like so:
OIFS=$IFS; IFS=$(echo -e "\n"); getopt $@; IFS=$OIFS echo -e " foo:\t$foo v:\t$v n:\t$n blah:\t$blah one:\t$one "
You might be curious about this line:
OIFS=$IFS; IFS=$(echo -e "\n"); getopt $@; IFS=$OIFS
IFS is the variable that tells bash how strings are separated (and mastering its use will go a long way towards enhancing your bash scripting skills.) Anyhow, by default IFS=” ” which normally is OK, but in our case we dont want “last arg” to be two seperate strings, but one. I cannot put the IFS assignment inside the function because by that point bash has already split the variable, it needs to be done at a level of the script in which $@ has not been touched yet. So I store the current IFS variable in $OIFS (Old IFS) and set IFS to a newline character. After running the function we reassign IFS to what it was beforehand. This is because I dont know what you might be doing with your IFS. There are lots of reasons you might have already assigned it to something else, and I wouldnt want to break your flow. So we do the polite thing.
And in case the above gets munged for some reason you can see the plain text version here: bash-getopt/getopt.sh
Anyways, hope this helps someone out. If not it’s still here for me when *I* need it 
Bash Coding Convention ../
We use dirname() a lot in php to make relative paths work from multiple locations like so. The advantages are many:
require dirname( dirname( __FILE__ ) ) . '/required-file.php'; $data = file_get_contents( dirname(__FILE__).'/data/info.dat');
But in bash we often dont do the same thing, we settle for the old standby “../”. Which is a shame because unless your directory structure is set up exactly right, and you have proper permissions, and you run the command from the right spot, it doesnt work as planned. I think part of the reason is that its not obvious how to reliably get a full path to the script from inside itself. Another reason is that ../ is shorter to type and easier to remember. Finally there’s always one time scripts for which this methodology is overkill. But if you’re planning to write a script which other people will (or might) be using, I think it’s good practice to do it right. Googling for things you’d think to search for on this subject does not yeild very informative results, or incomplete (incorrect) methods… so… here’s how to do the above php in bash:
source $(dirname $(dirname $(readlink -f $0)))/required-file.sh data=$(cat $(dirname $(readlink -f $0))/data/info.dat)
Hope this helps someone 
As a side note, the OSX readlink binary functions differently. You’ll want to use a package manager to install gnu coreutils, and iether use greadlink, or link greadlink to a higher precedence location on your $PATH (I have /opt/local/bin:/opt/local/sbin: at the beginning of my $PATH)
Simple TCP Daemon Example
Using some stuff I’ve covered in the past on my blog here’s a simple way to put up a daytime server (well to put any service onto a tcp port. I haven’t looked into its bi-directional capabilities yet, this was just sort of a proof-of-concept…
$ apt-get install ipsvd $ wget http://blog.apokalyptik.com/files/daemonize/daemonize.c $ cc daemonize.c -o daemonize $ ./daemonize /var/run/daytime.pid /var/log/daytime.log 'tcpsvd 0 13 date'
start/stop and/or monit script are an extremely short jump from there… And kind of trivial/menial… so I leave that as an exercise to you… if you care
Is it simple enough?
One question I sometimes forget to ask myself, when I’m coming up with solutions to problems is this: “Is it simple enough?” Even though I try and make a habit of asking myself that often I, sometimes, don’t ask often enough. The lesson of the importance of simplicity in both software development and systems management is one of those lessons that you really only ever “learn” after you’re war scarred and battle weary. One day it clicks, and you realize “hey… this should be easier.”
The interesting thing about the problems that we (as a community of tech minded developer types, and systems people) is that the majority of our solutions can be done easily. Sure there’s always something… at the deep dark core of a really difficult problem… that will always be that arcane black magic voodoo bit of code. But by and large everything else can (and should) be simple.
I wonder what stories you, my 2.5 readers, may have to share on the subject. When was it that you became totally fed up with writing complex solutions? Or have you some other opinion on the matter?