Using Wordpress for a Simple News Web Widget.

I often find myself needing a simple blog type news feed for the various sites I'm creating.  In the past when I was still doing my primary development on .NET I rolled my own solution using a SQL database to hold the entries.  At first I'd just put the data into the database directly but of course over time I wanted to be able to edit and add new entries from a web interface.  So I found myself incorporating TinyMCE and pretty soon I was writing a whole management interface just for editing news posts.  I knew it was time to stop myself before completely falling prey to NIH syndrome.

Lately I've been doing a fair amount of small projects using Wordpress and it seemed that I could just use its CMS to hold my news posts. I'd now have a repository for my posts as well as the ability to edit them.

In order to use the post data in my other sites I wanted to export my posts in JSON format. I've written a fair amount of JSON exporting code server side so I knew this wouldn't be a problem. Luckily I took a look in the Wordpress plugin repository and there's already a pretty nice plugin that dumps any post in JSON.

If you want to do this you first need to download and install the latest version of Wordpress.  This should take you all of 5 minutes unless you've never done it before in which case it might take you 30 minutes if you need to install LAMP or WAMP if you're on Windows.  You can also do it in about 5 minutes on an Amazon EC2 micro instance by following this excellent guide.

You will also need to download the JSON plugin and copy it to the plugins directory.  You can now create new posts in Wordpress and pull the data out in JSON. 

I'm using this for my Madtown yoga website.  If you look at the site you will see a list of  news posts on the right that are fed by the Wordpress blog.

The simple blog interface is here, where you can see a series of posts in the default wp skin. If you look here you can see the exact same posts rendered out as JSON data. Since I'm only using the blog as a data repository I didn't bother messing with the Wordpress UI. 

If you look at the HTML for the news section on the right you will only see two lines of relevant code.

<script src="/Scripts/wpnews.js" type="text/javascript"></script>
<div id="news-widget-container"></div>

I've implemented this as a web widget. A web widget is a way to allow people to very simply place some kind of functionality on a web page.  A container div with a specific ID is placed on the page and then that div is filled in on the client side by the corresponding JavaScript. If you want a good tutorial on web widgets you can check out this article.  It's the article I used to originally craft this news widget a few months ago.

In this case the JavaScript checks to see if JQuery v1.6.4 is currently loaded on the page and if not it loads it. It then reads the data from my Wordpress site as JSON into an array and then adds the posts as a series of divs to the HTML DOM . The actual formatting of the data is handled by a separate CSS file.

If you want to have some fun make a simple text document on your desktop that looks like this and save it as test.html.

<html>
<script src="http://madtownyoga.com/Scripts/wpnews.js" type="text/javascript"></script>
<div id="news-widget-container"></div>
</html>


If you then open this test.html file in your browser you should see an unformatted list of my news posts in the document you just created. If you add in the line

<link href="http://madtownyoga.com/Content/Site.css" rel="stylesheet" type="text/css" />

you will then see it with my formatting applied to the divs.  What's nice about widgets, besides the simplicity of using them, is the ability to their look on your page.  In this case add simple formatting to the classes FrontPageNewsTitle and FrontPageNewsContent and you can make my news feed look however you like on your web page.

If you take a look at the source code for wpnews.js it should be pretty straight forward to understand what I'm doing.  The majority of the code at the bottom is simply used to get JQuery loaded up if it isn't already used on the page.

A special note to be aware of when pulling data from a web site in JSON  format. Often when pulling data into a web widget the domain which has the data may be different from the domain from which we load our page.  The issue here is that JavaScript is not allowed to load JSON data cross domain. The way that you solve this problem is by using a format called JSONP. The only difference in JSON and JSONP is that in the latter case the data is wrapped in a function and then returned as an object when that function is called.  Since code can be loaded and executed cross domain you just got around the problem!

Raspberry Pi a la Node - Running nodejs on the Raspberry Pi

My local hackerspace Sector67 held a 24 hour build fest this weekend. Late last week in preparation for the event a few of us purchased some Raspberry Pi's.  If you're not familiar with the Pi, it's a credit card sized ARM based computer with a USB port, Ethernet port, GPU,  HDMI and Composite video out that runs Linux for about $35.  You can read more about it at the link above but the "overall real world performance is something like a 300MHZ Pentium 2, only with much, much swankier graphics."

I bought 2 with the plan to install nodejs on them.  I figured it'd be a super cheap way to create a web based controller people could use in other projects.  I'm happy to say it only took me about 3 days to make this a reality.  However it was pretty much 3 days of frustration that I'd like to keep the rest of you from having to experience so here's the scoop.

I bought the bare bones Raspberry Pi for $35.  When you buy the bare bones card you're going to need some more hardware to make it run. Here's a quick list:

1) Class 4 SD card from 4 gigs to 32 gigs in size.  The Pi uses this for its boot device.
2) A 5v 700mA micro usb power supply.
3) A powered usb hub.
4) USB Mouse/keyboard.
5) A monitor with HDMI input or an HDMI to DVI converter if your monitor is DVI only.
6) An SD card reader/writer to image the SD card with the linux boot image.

My Pi's arrived in the middle of the week along with the 2 32g SD cards I ordered off Amazon.  I rooted around my junk hardware closet and dug out an extra USB keyboard/mouse and an old Blackberry phone power supply that was 5V/650mA micro USB.  The one thing I realized I was missing was an SD card reader/writer.  I made a quick search on the internet and found that my local Target claimed to have a $10 cheap-o reader in stock.  Of course when I got there they didn't have it but they did have a $20 Belkin universal media writer as well as a couple of powered USB hubs on closeout for $5 each.  So I grabbed them and headed over to Sector67 to get things up and running.

The first order of business was to get the linux images on to my SD cards.  I decided to do this from my Windows based laptop so I downloaded the latest Debian image from the Raspberry Pi download page.  The actual release I picked up was the September 18th Raspbian "Wheezy" image.  After downloading and unzipping the image I was left with an .img file I needed to get onto the SD cards.  In order to facilitate this I needed a piece of imaging software called Image Writer for Windows.  After downloading and installing this I was able to quickly image both my SD cards with the Raspbian .img file.

Here's where I ran into my first problem.  I have a Dell monitor with HDMI input but I didn't have it at Sector.  I had an HDMI cable with an HDMI to DVI converter but when I plugged it in all I got was a blank screen.  I tried it with both Pi's and both of them did the same thing.  Ok fine, the Pi's also have a composite video output so I tried that next.  Same problem.  For some reason when I plugged both the composite and HDMI/DVI in at the same time I suddenly got video out through the HDMI/DVI combo.  If I unplugged the composite or the HDMI/DVI neither output would work.  If I plugged them both back in then the HDMI/DVI would work.  At least it allowed me to get working on the thing.  Later on I switched to my HDMI monitor with a straight HDMI connection and everything was fine.  However, I did experience some weirdness in that occasionally I would get a weird "puttering" sound through the HDMI audio.  This happened on both Pi's so I'm not quite sure what's up with that.

I was finally up and running with a basic Linux image on my Pi.  Now it was time to try and get nodejs up and running.  I did some looking around the web and the best article I could find was this one from July.  It looked highly promising.  Here's where I ran into my next gotcha.  While editing with VI I found that I couldn't enter a # character on my keyboard.  I got around it by cutting and pasting a # from somewhere else in the document but a few minutes later when I needed to use the | character on the command line I knew I was going to have to fix the problem.

The image that is on the Raspberry Pi website was obviously generated by someone in the UK as the keyboard settings are set for a UK keyboard and this is what was causing me grief.  To fix this issue you first have to type in the following:

$ sudo dpkg-reconfigure keyboard-configuration

You'll have to go through a few dialog boxes to select your keyboard and the region of US.  At one point you have to select Other to get to the other regions besides the UK ones.

After finishing you can reboot the system or just do the following:

$ sudo invoke-rc.d keyboard-setup start

Moving on with a functional keyboard I followed the instructions in the article and downloaded the 6.15 node distribution and attempted to get it compiled on the Pi.  So here's where things started to suck.  The first thing I found was it took anywhere from a 90 minutes to 2 hours to compile the v8 library on the Raspberry Pi.  I immediately, and by immediately I mean 90 minutes later, I ran into a problem when trying to link the library.  After searching around the net for a solution I found what I thought was my issue and recompiled.  90 minutes later I ran into another issue.  This went on for some time with trying different compiler options to get things working.  After 4 hours or so of frustration I decided to head home and work on it there.  Here came my next problem.  In my haste to head home I didn't bother issuing a

$ sudo halt

so when I got home my SD disc image was corrupted and wouldn't boot correctly.  Awesome.  So after a re-imaging of the card, upping the file system to use all the card, (I didn't mention this earlier but you need to do this to use all the space on the card and it takes like 30 minutes), and fixing the keyboard thing I was ready to go back at it.

Now for another woe.  The Pi has an option to overclock/overvolt it for better performance.  In the past this wasn't recommended and could void your warranty.  As of the latest release there is a turbo mode that lets you crank up your 700MHz processor in a variety of steps to 1GHz.  However they note that you need to "test stability" as it's dependent on your Pi and your power supply.

At this point I was getting pretty frustrated with compile times so I cranked that baby up to 1GHz, re-downloaded node and got to work compiling.  Unfortunately this ended really quick as for some reason the node compile died almost immediately.  I took a look at the file that wouldn't compile and found that the text file was corrupted.  Turns out that my "stability" was not good.  I assumed that my power supply was not up to the job, so I cranked my CPU back down to 700 MHz and reformatted again to get rid of my corrupt files.

I continued to get nowhere with building node and spent a lot of time searching for other ways to get it to build.  I probably tried 3 different articles that people had posted as well as one binary distribution that immediately died with a segmentation fault.  Finally on Saturday morning I came across this gist.  This is the one that made it all happen.  Down near the bottom is the post from a month ago with the script that worked for them as well as me:

export NODE_VER=0.8.7
cd ~
rm -rf ~/work/node-build/node-$NODE_VER;mkdir -p ~/work/node-build && cd ~/work/node-build
rm -rf ~/opt/node-$NODE_VER;mkdir -p ~/opt/node-$NODE_VER

curl -O https://github.com/adammw/node/commit/arm-patches-2.patch

curl http://nodejs.org/dist/v$NODE_VER/node-v$NODE_VER.tar.gz | tar xz
cd node-v$NODE_VER
git apply --stat --apply ../arm-patches-2.patch

./configure --shared-openssl --without-snapshot --prefix=~/opt/node-$NODE_VER

date;time make CFLAGS+=-O2 CXXFLAGS+=-O2 install;date

If you copy this into a shell script and run it you should get an actual build of node 0.8.7 and npm that works.  After the build I was quickly able to install express and have a web site up and running on the Raspberry Pi.  I was also able to install the serialport library for nodejs and talk to an Arduino board over the USB connection (a topic for another day).

I was so excited to have node running that I even went ahead and compiled redis for the Pi as well.
I just pulled down the distribution off this link  and did a

$ make -j 2

When it's done it will tell you to run "make test" which I did as well just for grins.  This takes quite a while to run but it passed everything for me.

I'm really looking forward to messing around with this more.  A super cheap, super tiny embedded linux system that is powered by nodejs has got me fired up to put all sorts of stuff online.  A HUGE HUGE tip of the hat to Adam Malcontenti-Wilson for the gist that got me up and running.

Finally here's the obligatory screenshot of my Raspberry Pi running the default Express installation on nodejs.

Yoga, Morse Code, and that Damn Song!

After an extended break I'm back to doing yoga 3 or 4 times a week.  I roll out for the 6 am class at Capital Fitness, grab a coffee on my way back from Pinkus Mcbride and by 7:15 I'm drinking it while catching up on the webs.

I really enjoy yoga.  It gives me an hour to center myself and pretty much changes my attitude for the day.  I can tell when I've been away from it.  It's hard to get other tech guys to try it out but luckily I know a few who have and they've become converts as well.

Last Monday I got up, got all my stuff together and headed out the door at 5:50 A.M.  It takes me 5 minutes to walk to the studio so my timing was perfect. I got down to the elevators and after pushing the button I heard it.  There was a faint electronic sound coming from somewhere that was going "Beep beep beep, beeeeeep beeeeeep beeeeep, beep beep beep".  3 short beeps, 3 long beeps, and then 3 short beeps.  Now maybe it's because I'm old or was a boy scout, but I recognized that immediately as SOS, the international Morse code cry for help.

I tried to track the source but couldn't get a bead on where it was coming from.  At the same time another tenant came out in scrubs obviously headed off to his job at a hospital somewhere.  I asked him if he heard the beeping and he said "Yeah it's been doing that for a while."  I said "Well that's Morse code for SOS" and he responded, "Whoa you're right, you should call our landlord or 911 or something."  He then jumped in the elevator and took off.

Now there is no way I'm making it to yoga.  There are people out there who will roll into yoga class late.  I'm not one of them.  It's just too disruptive and not good manners. I'm thinking calling 911 is not a great idea. The only solution is to call my landlord.

Our landlord is a pretty cool guy and he lives right above us in our building.  This is great since when something is wrong in the building it's typically fixed immediately.  I happen to have his cell phone so I decided it was time to wake him up.  I went back to our apartment and gave him a ring.  After a sleepy "Hello" I explained what was up and I'm pretty sure he was thinking "WTF, why are you calling me about this" but he agreed to meet me at the elevators in a couple minutes.

After hearing the beeping from the air vent he agreed that something was up.  We headed down a floor and couldn't hear anything, so we went back up a couple floors where the sound was loudly emitting from a service closet.  After unlocking it the cause was pretty apparent.  The drain for the building's air conditioner was clogged and water was pooling in the closet.  My landlord had put a moisture meter in the room for this very occurrence and it was sending out the "SOS" message over and over.

I was a little pissed at this point.  Wouldn't an annoying electronic "whoop whoop", or siren, or anything other than "SOS" have been just as effective?  Perhaps not as it probably wouldn't have stopped me in my tracks on the way to yoga if it had been anything else. But still......

Later in the day I sent him an email apologizing for getting him out of bed at 6 A.M. but that it would make a good blog post. His response was

"I was almost up anyway and better safe than sorry.  
You're right--you are likely the only person who knows the SOS sequence. 
Please be sure to list how quickly McBride responded too."

So if you're ever looking for an apartment know that McBride Companies has awesome response time to apartment issues, even if they have annoying Flash Music on their website. Seriously go listen to it.

Thursday morning I went off for my usual 6 am Yoga class.  Before the class got rolling our instructor Erin said we'd be working on opening up for Hanumanasana, also known as the splits, not exactly my forte. She also said she had put together an inspirational playlist for the day.

Now  if you asked me to name what tranquil yoga song we'd be starting off with I can guarantee you that my first 10,000 guesses would not be the song that haunts my life, Europe's Final Countdown.  If you remember, it was only a few months ago I last ran into this song outside my bedroom window.  When it came on I pretty much lost it. I was laughing so hard I couldn't even pay attention to what was going on.

As a bonus here's a couple of pretty decent yoga type tunes relating to the Monkey God,  Hanuman. One is by MC Yogi and the other one by Wah.

Namaste!

Sherman Hemsley - Peace

Sherman Hemsley left us yesterday and that sucks.

In the late 90s we were releasing our product ACID to the market and getting ready to hit COMDEX and NAMM (our largest trade shows).  One day our head of marketing Roy Elkins came up to me and said that he was considering using Sherman Hemsley as our spokesperson at the tradeshows.  Roy had been long time friends with Sherman and what many people didn't know was Sherman was a musician.  Roy had hooked him up with a copy of ACID and Sherman was loving making his own music with it.

At COMDEX we had these giant laminated 10 foot tall head shots of Sherman.  When we came back I remember that one of those hung on the wall of our shipping department for years.  Every day you'd roll through there would be a giant Sherm staring at you.  There was something right about that.  One of the guys from that shipping department works in sales today at Sonic Foundry.  A couple years ago I asked him if he knew whatever happened to that poster. He wasn't sure and we both were a little bit bummed that it'd been lost.  I have know idea where I'd ever hang it but I really wish I still had it.

A good friend of mine Steve who worked for Creative Labs, the makers of the Soundblaster card, came up to me one day during the convention.  As he stood looking over at Sherman he said, "You guys really have Sherman Hemsley as your spokesperson for ACID.  Man, I love you guys, you're so weird!" He then quickly rushed over to meet him.

Later that evening Roy, Rimas, and I were sitting at a black jack table in the casino of our hotel.  Sherman had told Roy that he wanted to come down and hang with us for a while.  It was pretty apparent when Sherman had hit the casino.  I swear that every single table in that casino turned and looked when he went by.  You could hear people saying "Hey! isn't that George Jefferson?".

I didn't get to spend a lot of time with Sherman, but from what I did I noticed a humble, sincerely nice man.  His voice always quiet and a huge smile as he met with adoring fans.  He loved music, and he loved making music.  I am lucky to have met him.

I have a lot of mementos from my days at Sonic Foundry, but this is one of my favorites.  At the shows we handed out CD's with 3 tracks that Sherman had created using ACID.  His vocals are buried quietly in the tracks, much like the gentle soul that he was.  His simple message of "peace", so representative of the man.  So where ever you are are Sherman Hemsley, "Peace".

On my way!

I'm "On my way!" to Mexico for my annual house building trip.  Each year I join my dad and about 100 others to build 5 houses outside Tecate.    It's always a test in connectivity since we stay at a small ranch with lots of bunk beds and iffy wireless.  It's also a serious test of my rudimentary Spanish.

I fly out to San Diego a day ahead of time to buy tools and such so last night I dropped in a local Mexican restaurant to grab some food.  While I was waiting I heard a guy say to his girlfriend/wife "Not only are you ruining my life but you're ruining my meal as well".  Classy.

Can someone tell my why Apple decided that the auto-correct in the IPhone should change "omw" to "On my way!".  Really Apple?  We don't all know what omw means?  Someone there decided that we should all look super cheery with "On my way!". I don't know why this bugs me so much.

Nos vemos!

Automatic Teller UI, Can we fix this please?

While sitting in a restaurant last night I was watching the waitress use the touch screen POS system.  I'm always interested in the user interface of systems like this, and though it certainly seemed functional enough, I have to say that the button look and layout was just plain ugly.  I could only think "would it really have been that hard to pretty this thing up a bit."

Earlier in the day I had stopped at my local Chase ATM to grab some cash.  The woman in front of me finished up her transaction and rushed off.  When I approached the screen I was met with the "Do you want another transaction?" screen.  While I was pressing the "No" response, I thought "Thank God I'm not somebody who would extract cash from this person's account".

Which brings me to one of my biggest User Interface pet peeves.  Why does this screen exist?  What programming genius decided that every person who ever uses an ATM should be asked if they would like to perform another transaction?  I have got to believe that literally 99% of all users walk up to an ATM and perform a single transaction.   Whoever designed this decided that it was better for 99 people to have to press NO, taking the chance that they walk away and have money stolen from their account, for the convenience of the 1 person who does multiple transactions.

Some bank web-sites even mention this in their ATM security section.  And I quote:

"Make sure you have finalized the transaction and removed your card before leaving the ATM. Some ATMs ask if you want "another transaction". You must reply "No" to close out your card's transactions. If you do not close your card's transactions and remove your card, the next ATM user can use your card."

The simple answer here is to do away with this screen. Let the 1 out of 100 users who actually wants to do a deposit and get some cash re-swipe his card and reenter his PIN.

Occasionally you still run into ATMs that have the "suck your card inside mechanism."  A number of times I've watched people walk away counting their money leaving their card inside the machine.  I never understood why those machines don't kick out the card and then dispense your money. No one is ever going to leave without their money.  It would've been so simple to just change the order of cash dispersion and card ejection and problem solved.

As a programmer these kinds of things make me nuts.  I'm hoping that the ATM industry did lots of research and there's something I'm missing here. There is some mysterious reason why they chose to program these machines in this fashion.  My guess is it was someone not thinking about it, or someone who did, but thought that fighting to get the change made late in production was too much of a hassle.  No matter what the reason it's not helping out the poor person who doesn't have an honest person behind them when they forget to press No.

Using Webstorm or PHPStorm and Mocha for unit testing in Node.js

I have been working on a node.js project and decided I wanted to get some unit testing going .  I did some research and there are a lot of test frameworks out there for node.  You can use Jasmine, node-unit, Expresso, Mocha or a myriad of other frameworks.  Apparently this is a problem people have decided to solve over and over and over.  After a bit of research I settled on Mocha.  It’s written by the same guy who writes the awesome Express library and is well supported. 

You can write your unit tests in a variety of formats whether you like BDD or TDD and you can write them in either javascript or coffeescript.  Mocha is pretty deep and supports all sorts of input and output formats. There is actually a great tutorial on peepcode called Full Stack Node JS that goes into a lot of stuff on getting started with node and has a lengthy section on testing with Mocha. Spend the $12 right now it’s worth it. 

After you get done watching that you should be able to write some basic unit tests in javascript or coffeescript and run them from the command line using npm test.  I say “should” because like all things “node”, documentation is pretty sparse.  Be prepared to do some googling. 

So now that I was able to write unit tests and run them from the command line I should've been happy as a clam right?  Wrong.  Typically when I’m writing code I like to write a test method and then run it in a debugger.  I did some searching around and couldn't fine a way to do this with Webstorm, my choice of javascript editor.  I like Webstorm because you can debug your javascript from within the IDE and single step through the code. If you need to do PHP stuff you probably want to buy PHPStorm as it has all of the functionality of Webstorm plus the ability to work with PHP.  I bought Webstorm first but 6 months after the fact the people at JetBrains were nice enough to give me a discount when I upgraded to PHPStorm.

When you run the Mocha test framework it runs a node.js program that takes a list of test files on the command line.  I realized that if I could figure out a way to pass the files without having to use the command line I could probably debug my js files that are loaded by Mocha.  My plan was to run the mocha.js file from Webstorm in debug mode and that would then load up my test js files. After spending a few hours digging into the Mocha code, it looked like the plumbing was there for what I needed but it was still going to take some time to figure out exactly what to do.

I don’t know why but for some reason I decided to poke around a bit on TJ Holowaychuk’s blog (the author of Mocha).  I clicked on “About” which took me to the footer of his blog. By coincidence the last blog entry on the page from March 24^th said:

“A new JS API was added, which mocha(1) now utilizes. This higher-level JS API will make it easier for those who want to script the testing process with Mocha. I have yet to document this API but it looks like this:”

There it was right in front of me.  20 lines of code that would help me do exactly what I needed to put together my own “runner” file for doing my unit tests in a project.  Holy stroke of luck! I found the “I have yet to document this API” statement to be quite humorous since you rarely find heavily documented node.js libraries.  Here’s what I’m doing to make my test files Mocha compatible as well as debuggable from within Webstorm.

First add Mocha to your node project by doing the usual "npm install mocha" and make a directory called test.  This is where all your tests will live.  You can make subdirectories for your tests if you’ve got lots of them but make sure everything lives in test.  In test you create a base file called testRunner.js.  This file has the code you need to load your other tests and it is the file you will run in debug mode from within Webstorm.  

Your project should look something like this:

The code for testRunner is as follows

var testFiles=["test/_helper.js","test/tests.js","test/tests.coffee"];


var Mocha = require('mocha');


var mocha = new Mocha;


mocha.reporter('spec').ui('bdd');


for (var i =0;i<testFiles.length;i++){


 mocha.addFile(testFiles[i]);


}

var runner = mocha.run(function(){

                console.log('finished');

});

The array at the top is where you put the names of the files that contain all your tests.  As you can see above I’m going to run 3 test files named _helper.js,tests.js, and tests.coffee.  The _helper.js is stub file and all it contains is the follow line of code:

require('coffee-script');

If you’re not going to write any of your test scripts in coffeescript then you don’t need to include this file.  If you are, then make sure it’s the first file in your list of tests.

The next lines of code load Mocha, create an instance of it, specify that you want to use the behavior driven development style of tests, add the test files, and then run them. When you run this from Webstorm the output from your tests wlll show up in the Webstorm console window and you can single step through the code for debugging. 

I hope this helps some of the other people out there who were trying to figure out how to work with Mocha and Webstorm.