I want to share an experience with you all, since everybody loves to watch a train wreck. As developers, we like to think that the state of the LinuxTV project is continuously improving. Then I have days like yesterday….
I wanted to build myself a MythTV box for personal use. Since my goal is to use this “in production”, I figured I would pick a PCI based tuner that had mature driver support. I also assumed that this would minimize the likelihood of me running into trouble. I ended up choosing the PCTV 800i, which is a cx88 card that uses the xc5000 tuner and s5h1409 demodulator. It also has an onboard IR receiver, something which for once I actually cared about since I planned on putting this in my living room. It’s also worth mentioning that the cx88 driver provides RC events via the input layer for the remote control that ships with the PCTV 800i’s, it should “just work” without having to muck with lirc.
I built a box with a fresh install of Ubuntu 9.10, and the card was detected “out-of-the-box”. No digging around the Internet for firmware was required since everything was already included in the distro (including the updated xc5000 firmware KernelLabs got merged last year). I ran Synaptic and installed the MythTV packages, ran MythTV setup, and in a matter of minutes was watching an ATSC stream. Mythtv-setup found the card, the channel scanner worked as expected, and I was happy. The only thing remaining was to get the remote control working – and that’s when everything went downhill.
People often approach me about a problem with their tuner, and my first advice is always “update to the latest code and see if it’s still a problem”. Hence it was only natural that I should take my own advice. I pulled the latest v4l-dvb code from linuxtv.org, rebooted, and my PC hung on startup. I tried various boot combinations (such as single-user mode), and finally resorted to cracking open the case and removing the card in order to get the PC to boot. At that point, I was able to see the /var/log/messages from the failed boot attempts, which included an kernel OOPS in the IR initialization.
It’s probably worth noting at this point that I have been working on various projects for the last few weeks that all involved branches, so I had not running the v4l-dvb trunk for almost a month. So, as a test, I updated to the trunk on my laptop and plugged in my HVR-950 – and hit the same OOPS.
Somehow a regression was introduced into the trunk which causes an OOPS on both cx88 and em28xx based devices whenever the driver is loaded. The LinuxTV maintainer apparently has decided to use the v4l-dvb trunk as his personal sandbox, instead of putting his experimental changes on a branch where others can test them before they get merged into the mainline. To add insult to injury, this problem was reported by five different people with five different products on the mailing list over the last four weeks, and the one line patch still hasn’t been checked in.
Once I added the patch to my v4l-dvb tree, the driver started working again, but the IR support still didn’t work. I plugged in my HVR-950 and confirmed that the IR was working there, suggesting some difference between the two. I spent time comparing the way the IR subsystem was initialized by the two drivers, and didn’t see anything obvious. I then started added printk() calls to the IR code, and concluded that both devices were sending the same key sequence to the input subsystem, therefore the keys must be dropped somewhere in the input subsystem.
So now I have to download the full 2.6.31 kernel source so I can add debugging to the input subsystem and try to figure *why* the commands are being dropped. After several hours of littering the input subsystem with printk() calls, I have concluded that in fact the keys are being delivered to the keyboard driver, but not to the event interface because the input device handle is not in an open state (at which point I realize that if I drop out of X11 to the virtual console, the remote control works). More printk() statement and dump_stack() calls added to the input_device_open() calls show that on the em28xx device the event interface handle is opened by the hal daemon, but not in the case of the cx88 device.
So, now I’m downloading and starting to look at the source code and logs for the hal daemon. By looking at the logs, and correlating them against the source code (since the quality of the logging isn’t good enough to figure out what is going on *without* the source code), I determine that for some reason hald is not binding to the input device because the cx88 IR device is set to explicitly be ignored by hal. More digging through code reveals that there is a blacklist being read. Finally, I arrive at the following:
/usr/share/hal/fdi/preprobe/20thirdparty/lirc.fdi
<?xml version="1.0" encoding="UTF-8"?>
<deviceinfo version="0.2">
<device>
<match key="info.product" contains_ncase="saa7134 ir">
<merge key="info.ignore" type="bool">true</merge>
</match>
<match key="info.product" contains_ncase="IR-Receiver">
<merge key="info.ignore" type="bool">true</merge>
</match>
<match key="info.product" contains_ncase="cx88 IR">
<merge key="info.ignore" type="bool">true</merge>
</match>
<match key="info.product" contains_ncase="bttv IR">
<merge key="info.ignore" type="bool">true</merge>
</match>
<merge key="info.ignore" type="bool">true</merge>
</match>
</device>
</deviceinfo>
Yes, under Ubuntu, they have explicitly told hal to ignore cx88 based input device, but *not* to ignore em28xx based input devices, which is why the two behaved inconsistently. Presumably they added cx88 to the blacklist to avoid conflicts with lircd.
In summary, removing the one line from that blacklist file will result in the device being found by HAL and the remote control will start to work. But look at what I had to do to come to that realization? I’m a developer who has worked on LinuxTV 20-30 hours a week for the last two years, and it took me an entire day to make this work (after hours of digging through the source code to v4l-dvb, the Linux kernel, and hald). Imagine if I had been a regular user? I probably *never* would have gotten it working.
As developers, it is important to occasionally take a step back and look at the things we are doing poorly and how we can improve. It’s also important to eat our own dog food occasionally to get some perspective in what end-users actually experience attempting to use our software.