I don’t care what Google or others say, this isn’t open source. This is published source. There is no way to see bugs, contribute a patch, or take part in discussions on development. The only saving grace is that Google is publishing their code with a fairly unrestrictive license.

What this situation is screaming for is to have someone with a desire to actually behave in an open source manner to come along and fork the code. Then allow developers to contribute to that branch and let Google go it on their own. My hope is that Amazon will do exactly that with their fork of the Android code base.

Exception in thread “Multicast Heartbeat Receiver Thread” java.lang.NullPointerException
at org.slf4j.impl.Log4jLoggerAdapter.error(Log4jLoggerAdapter.java:493)
at net.sf.ehcache.distribution.MulticastKeepaliveHeartbeatReceiver$MulticastReceiverThread.run(MulticastKeepaliveHeartbeatReceiver.java:137)

After the error, the Multicast Heartbeat Receiver would not shutdown and would just hang. Eventually you would have to kill off the java process manually.

Doing a quick search found this page. With a small snippet added to the web.xml to add a shutdown listener everything shutdowns cleanly now.

<listener>
  <listener-class>net.sf.ehcache.constructs.web.ShutdownListener</listener-class>
</listener>

One nice bonus of running ReadPath, is that it is constantly spidering content from RSS feeds for the news reader. The other day I realized that I’ve now stored a full billion content items going back several years. So of course having that much content I had to create a search engine to mine it. So, I created MiniSearch to play with different concepts involved in running a general search engine. There are a lot of things that turned out to be a lot harder than expected.

Currently the index is in the process of being built and only includes 20% of available content. There is also a lot of work to be done with ranking still. I’ll post again when I think it’s in a more usable state.

Great Job Mozilla guys

One of the speakers, Mr. Brodhun, is uniquely qualified on this subject having previously served as Technical Director for Enterprise Standards and Technologies for the United States Marine Corps, where he oversaw the deployment of approximately 2,300 ESX hosts and nearly 7,000 virtual machines across 167 sites.

Regardless of the size of your virtualization project, you’ll learn how to maximize uptime and performance of mission critical applications, while eliminating hidden costs that can decrease virtualization ROI upwards of 50%.

Click here to register for this webinar.

Then if you find a RSS feed that you’d like to add to ReadPath, drag the link below to your bookmark bar.

Add To ReadPath

Then when you’re on a page that you want to subscribe to, press the bookmarklet and it will have ReadPath subscribe you to the feed.

My production cluster is still running HBase 0.20.3 and has performed flawlessly. It has a table holding half a billion content items, taking up several terabytes of space, and has made it through several disk failures without a hitch. However, I’m looking at the Cloudera distro because I’m not happy with having to repackage everything, test it out, push to the live cluster, and then retest to make sure that everything made it properly every time a new release comes out. I’m hoping that using the Cloudera distro will simplify a lot of this. I’m also hoping that with the patches that they include and testing being done that I’ll have a more stable cluster. I had a real bad experience with the 20.4 update which is why production is still on 20.3.

One major problem that I still have, even with the Cloudera distro, is that the LZO code isn’t included due to licensing problems. I’m really hoping that the new compression options can be packaged up soon so that these libraries don’t need to be maintained separately any more.

A couple quick notes that I found from my testing.

• The way that HBase interacts with and manages zookeeper has changed. It’s more like running an unmanaged zookeeper setup. I found that not only did I need to make sure that the zookeeper configs in the hbase-site.xml needed to be correct on all of the servers, but that when I ran map-reduce jobs against HBase that it seemed to be reading from /etc/zookeeper/zoo.cfg and that this needed to be correct on all of the regionservers. I initially had only edited it on the server running my zookeeper instance. I also added the file to the HADOOP_CLASSPATH in hadoop-env.sh but I’m not sure that that’s required.
• I wish that there was a better way to manage the HADOOP_CLASSPATH and it’s inclusion of the hbase and zookeeper jar files. I’m trying to find a way so that this doesn’t need to be edited each time I update the software to a new version.
• I had to change the value for dfs.datanode.socket.write.timeout. On the live cluster I have it set to 0 which is supposed to mean no timeout, but it appears that there is a bug with this latest version that doesn’t respect that value properly. So I just set it to a high number.

One of the difficulties with VMware is that their acronyms can be very difficult to wade through. ESXi is what they refer to as a hypervisor. This essentially is a very cut down operating system that is designed to only run other Virtual Machines. There are some requirements to running ESXi, I had to go through 3-4 servers before I found one that the installer had all of the drivers. I finally got it to run a server I had picked up from Penguin Computing (2x dual core Opteron with 4Gb mem and 250Gb hard drive).

Once I found a server that worked, the system installed quickly. The next problem was that you need to download the vSphere client to administer the server which is windows only (there are command line clients for other operating systems, but I wasn’t ready for that yet). I didn’t have a windows box laying around (all linux and mac), so I had to launch a WinXP VM in workstation on my linux desktop to administer my ESXi server. Amazingly everything worked great.

The next issue that I ran into was that I already had a large number of VMs created that I was using on workstation, but I couldn’t see how to get them on to the ESXi server. In the vSphere client there are clear instructions on how to create a new VM or download an appliance, but not how to import an existing VM. It turns out that VMware has a very simple way of doing this using the VMware Converter. This product works as a switchboard allowing you to convert or move VMs from one place to another, a really handy tool.

Overall ESXi is a great tool for running a whole bunch of server VMs. VMware offers a huge number of management products in the vSphere product line for managing load and moving VMs in a datacenter. But if you just need to run a few VMs on a single server I would definitely recommend looking at ESXi.

• Often the mail icon will show that I have new mail, but when I open the app there is nothing there, even after hitting the refresh button.
• After I delete email they’ll show back up again as unread.
• I’ll often get a list of blank emails that I can’t do anything with.

Closing the Email app has no effect. The only thing that has helped so far is to double click the button to get to the quick switch screen, hold down the mail icon until they vibrate and the minus icon appears (I believe that this kills the background process), click the minus’ to close all running background processes. Once I do this Mail will begin to sync again. This leads me to believe that there are issues with communication between the background process and when you bring the app to the foreground.

I’m checking every day to see if Apple has pushed an update, but this was a pretty big bug to let slip into ios4.

There seems to be some healthy competition again in the browser space and Chrome is pulling ahead with the JavaScript performance. I had stuck with Firefox for a long time because it had Firebug, which is probably the single most useful web development tool out there. But now that JavaScript libraries like YUI are getting so much more solid and stable I’m finding that I don’t need to dive in with Firebug as much.

Overall I’m just happy to see that I have a choice. While performance and stability are great, I hope there’s more competition among the browsers as web application platforms and not just web content distribution. There’s still a long way to go in this area.

Goldman Sachs as a firm has internal views into almost every market there is. Seeing both sides of trades as well as the demand for creation of new products. The Manager would not say that internal talk among different groups wasn’t used in the determination on how to handle products that Goldman itself held. He squirmed out of taking any position.

The clear implication that you can take from these different data points is that Goldman Sachs has the ability to hold any product that they like and sell those that they don’t. They are not a neutral market making entity. They don’t see selling products that they don’t like to their own clients as a conflict of interest. Customers of Goldman Sachs need to ask themselves why the firm is selling any product. Because, if the item up for sale was worth holding, Goldman itself would hold it and not sell to the customer.

Goldman Sach’s ability to take positions and to be a market maker should be stripped. There should be a requirement that they can’t profit by taking positions as a market maker. There is no way to separate the conflict of interest otherwise.

The software is still listed as beta and definitely has a rough around the edges feel. But it does all of the basic things that kindle for the iPhone does. It allows you to pull in any of the books from your amazon archive, sync locations between different devices, change the font and reading formats to fit the device, and sync notes between devices. The main disappointment that I had was that the mac version doesn’t appear to have a book search feature. The laptop version of the software is the ideal place to be able to go back into your books and take notes or look up relevant parts. Without a basic search feature though this is severely hampered.

Up until now I’ve been splitting my two basic classes of book purchases, books that I’d just like to read from cover to cover and books that I need as reference material. I still purchase in physical form most of my computer books because I want to be able to look something up quickly and flip to exactly the point that I need. This is not an ideal use case for the kindle though, which is really good at reading from cover to cover. If the laptop software had a good search feature I would definitely start looking at purchasing reference books in digital form. It would be a great tool for looking things up and taking notes.

Well, hopefully Amazon listens and gets search implemented. It’ll transform the laptop software from a curiosity into a really powerful tool.

I had originally placed a few types of ads on these pages, just so that I could try them out somewhere other than at the day job. But now that I’ve got ReadPath cranking there really isn’t any need to experiment here any longer.

I’m fairly happy with this new theme, but you might continue to see some changes as I customize things.

A few months ago, I downloaded Amazon’s kindle app for the iPhone, more out of curiosity than anything else. Initially I just couldn’t see myself reading books on such a small screen, especially one that was backlit and would cause eye-strain. However, in the last couple weeks I’ve discovered how seamlessly Amazon has made syncing devices and reading locations. Since during the day, I pretty much always have the iPhone with me, I can just pull it out and start reading. The iPhone is perfect for the situation where you might only have a few minutes and might need to put it away very quickly. I was getting my van washed yesterday and was able to read for 5 min at the car wash. The magic happens though when you get back home, turn on your kindle and it knows to sync your reading location up to where you left off on the iPhone and vice-versa.

I’m now finding that I’m able to get even more reading done during the day. Filling in small breaks instead of waiting for a single block of reading before bed.

I was on the lookout for a really great notes/task management tool that would sync back and forth between my laptop and iPhone. I tried out several of the web based ones and Evernote(which has a desktop and iPhone app), but they never quite did exactly what I needed.

Then I found Things. It’s a bit expensive, but it got great reviews, so I thought I’d try it out. You have to purchase the desktop app and the iPhone app separately, but they work great together. Syncing whenever you have wifi turned on on the iPhone. You can quickly create notes, set them as scheduled, inbox, or whatever you want. Update them and move them around easily. The desktop app even integrates in with iCal’s todo list.

If you need a fast way to keep track of a mix of small and large things and the ability to sync back and forth from phone to laptop, I would highly recommend checking out Things.

What I really wanted was to be able to just push or pull the file from my iPhone to the desktop over my local wifi. A really big first step getting there was finding the app Air Sharing.  This allows you to expose a portion of your disk on the iPhone and share it over wifi through bonjour or http. Great app, it’s got all of the controls that you need for security. Also, you can log into other servers over ssh, ftp, ftps, … browse to the file you want, and then either copy it locally or mail it off(This is great since the default iPhone is really cranky about mailing large files). You can even select to mail off an entire directory and it will make a nice zip of the directory for you before emailing it.

Air Sharing got me most of the way to where I wanted to go, but I still couldn’t figure out how to get the items that I wanted out of my Camera Roll and into the shared directory. Then I realized that when you’re in the “My Documents” area of Air Sharing and you click “Edit” and then “+” that there is an option to add from Photos. This allows you to select whatever you want from your Camera Roll and add it to the shared dir in “My Documents”.

Now when you add the iPhone as a new server in your finder, you’ll see the videos that you selected in the folder just like it was any other shared file on a remote server.

My gripe though is that appears that in the last several months that John C. Dvorack has become an almost regular guest, appearing on the show pretty much every week. When Dvorack first started appearing on the show, he could tell some interesting stories about early happenings in tech and would keep things moving if the topic got too far afield. Lately though all he does is destroy any discussion that does get going. Not sure why the change happened, but he seems to only want to talk about himself and will not tolerate discussion of things that aren’t of personal interest.  His presence is completely ruining the show, to the point that if I hear him introduced I’ll move on the next podcast.

The latest TWIT is as good an example as any, every time the other guests would get into a substantial discussion on a topic, Dvorack would blunder in with some random unrelated thought that he had and derail the discussion. The beauty of podcasting is that there isn’t the time slot constraints that other mediums have, so if you find something worth discussing you can flesh it out.

I love the work that Leo does and listen to hours of his programming each week, but I’m afraid that he might have too much personal loyalty to Dvorack to see what’s happening.  I can’t imagine how Leo’s not completely burned out, but maybe as a New Year’s resolution he needs to take a fresh look at his programming and get back to basics a bit.  You just can’t lose with good guests and good discussions.

I was up till 1:30am last night reading and am having serious trouble moving today. All of this could have been avoided if I had known that I had been reading for 4 hours straight.

Of course you could say that I should just put a clock in the family room and solve the problem, but then I’d have to go grab a new clock and that’s not nearly as easy as writing a blog post.