Building a MythTV box

I want to share an experience with you all, since everybody loves to watch a train wreck. As developers, we like to think that the state of the LinuxTV project is continuously improving. Then I have days like yesterday….

I wanted to build myself a MythTV box for personal use. Since my goal is to use this “in production”, I figured I would pick a PCI based tuner that had mature driver support. I also assumed that this would minimize the likelihood of me running into trouble. I ended up choosing the PCTV 800i, which is a cx88 card that uses the xc5000 tuner and s5h1409 demodulator. It also has an onboard IR receiver, something which for once I actually cared about since I planned on putting this in my living room. It’s also worth mentioning that the cx88 driver provides RC events via the input layer for the remote control that ships with the PCTV 800i’s, it should “just work” without having to muck with lirc.

I built a box with a fresh install of Ubuntu 9.10, and the card was detected “out-of-the-box”. No digging around the Internet for firmware was required since everything was already included in the distro (including the updated xc5000 firmware KernelLabs got merged last year). I ran Synaptic and installed the MythTV packages, ran MythTV setup, and in a matter of minutes was watching an ATSC stream. Mythtv-setup found the card, the channel scanner worked as expected, and I was happy. The only thing remaining was to get the remote control working – and that’s when everything went downhill.

People often approach me about a problem with their tuner, and my first advice is always “update to the latest code and see if it’s still a problem”. Hence it was only natural that I should take my own advice. I pulled the latest v4l-dvb code from linuxtv.org, rebooted, and my PC hung on startup. I tried various boot combinations (such as single-user mode), and finally resorted to cracking open the case and removing the card in order to get the PC to boot. At that point, I was able to see the /var/log/messages from the failed boot attempts, which included an kernel OOPS in the IR initialization.

It’s probably worth noting at this point that I have been working on various projects for the last few weeks that all involved branches, so I had not running the v4l-dvb trunk for almost a month. So, as a test, I updated to the trunk on my laptop and plugged in my HVR-950 – and hit the same OOPS.

Somehow a regression was introduced into the trunk which causes an OOPS on both cx88 and em28xx based devices whenever the driver is loaded. The LinuxTV maintainer apparently has decided to use the v4l-dvb trunk as his personal sandbox, instead of putting his experimental changes on a branch where others can test them before they get merged into the mainline. To add insult to injury, this problem was reported by five different people with five different products on the mailing list over the last four weeks, and the one line patch still hasn’t been checked in.

Once I added the patch to my v4l-dvb tree, the driver started working again, but the IR support still didn’t work. I plugged in my HVR-950 and confirmed that the IR was working there, suggesting some difference between the two. I spent time comparing the way the IR subsystem was initialized by the two drivers, and didn’t see anything obvious. I then started added printk() calls to the IR code, and concluded that both devices were sending the same key sequence to the input subsystem, therefore the keys must be dropped somewhere in the input subsystem.

So now I have to download the full 2.6.31 kernel source so I can add debugging to the input subsystem and try to figure *why* the commands are being dropped. After several hours of littering the input subsystem with printk() calls, I have concluded that in fact the keys are being delivered to the keyboard driver, but not to the event interface because the input device handle is not in an open state (at which point I realize that if I drop out of X11 to the virtual console, the remote control works). More printk() statement and dump_stack() calls added to the input_device_open() calls show that on the em28xx device the event interface handle is opened by the hal daemon, but not in the case of the cx88 device.

So, now I’m downloading and starting to look at the source code and logs for the hal daemon. By looking at the logs, and correlating them against the source code (since the quality of the logging isn’t good enough to figure out what is going on *without* the source code), I determine that for some reason hald is not binding to the input device because the cx88 IR device is set to explicitly be ignored by hal. More digging through code reveals that there is a blacklist being read. Finally, I arrive at the following:

/usr/share/hal/fdi/preprobe/20thirdparty/lirc.fdi

<?xml version="1.0" encoding="UTF-8"?>

<deviceinfo version="0.2">
  <device>
     <match key="info.product" contains_ncase="saa7134 ir">
        <merge key="info.ignore" type="bool">true</merge>
     </match>
     <match key="info.product" contains_ncase="IR-Receiver">
        <merge key="info.ignore" type="bool">true</merge>
     </match>
     <match key="info.product" contains_ncase="cx88 IR">
        <merge key="info.ignore" type="bool">true</merge>
     </match>
     <match key="info.product" contains_ncase="bttv IR">
        <merge key="info.ignore" type="bool">true</merge>
     </match>
        <merge key="info.ignore" type="bool">true</merge>
     </match>
  </device>
</deviceinfo>

Yes, under Ubuntu, they have explicitly told hal to ignore cx88 based input device, but *not* to ignore em28xx based input devices, which is why the two behaved inconsistently. Presumably they added cx88 to the blacklist to avoid conflicts with lircd.

In summary, removing the one line from that blacklist file will result in the device being found by HAL and the remote control will start to work. But look at what I had to do to come to that realization? I’m a developer who has worked on LinuxTV 20-30 hours a week for the last two years, and it took me an entire day to make this work (after hours of digging through the source code to v4l-dvb, the Linux kernel, and hald). Imagine if I had been a regular user? I probably *never* would have gotten it working.

As developers, it is important to occasionally take a step back and look at the things we are doing poorly and how we can improve. It’s also important to eat our own dog food occasionally to get some perspective in what end-users actually experience attempting to use our software.

8 thoughts on “Building a MythTV box

  1. I think everyone agrees that testing is critical to development. Basic rules (like testing in an isolated repository before committing to trunk) should be in place, but ideally some automated regression testing tools would be developed for major projects involving many moving parts.

    We can see some of this going on here: http://www.phoronix.com/scan.php?page=article&item=linux_perf_regressions&num=1

    Great post Devin – and good luck with the MythTV box!

    • Hello Andrew,

      I think the challenge in most cases is dealing with the fact that MythTV is entirely a volunteer project. As a result, the few developers who do contribute work on the areas that interest them. Things like regression testing and documentation aren’t very sexy, and thus they don’t get the priority. In commercial product development, these are the things that you pay less experienced people to do because they tend to be boring and monotonous.

      Let’s also not forget that test environments aren’t zero-cost. In a commercial product environment, it’s easy when you have a dedicated test lab and a shelf with every possible tuner card, access to test equipment and different signal sources, and a variety of configurations at your disposal. For a project like MythTV, in most cases developers have access to the limited subset of hardware that constitutes their environment, so you end up with situations where code is well tested with the tuner card and version of Linux that the developer had (and not much else).

      And as volunteers, you cannot really tell them what to do. You cannot set their priorities in terms of dictating what bugs are important to you, or what development style you would prefer they employ. In fact, I see alot of things that happen in open source projects that I simply wouldn’t tolerate in a commercial environment – I’ve seen developers do things that I have fired employees for doing.

      The moral of the story, IMHO, is that you get what you pay for. I certainly don’t fault the developers personally for the state of affairs, but I do think that users have to set your expectations appropriately.

      Devin

  2. I can understand the frustration! Getting a Mythtv box up and running well enough can be soul crushing. I remember when I installed .8 I had to install with a regular LCD, then, after rebooting from the initial install, remove the LCD and plug in the Component Out…just to get video.

    Aside from better developer discipline and showering new hardware on all devs to test their code out, you’ve done the next best thing. You described your problem, included the technical situation, described how you worked it thru and then presented a solution.

    With the right indexing and search optimization, your solution’ll get picked up by the next person who has it and searches for help, meaning any above average user will be able to solve it in a fraction of the time.

    In fact, searching “HVR-950 IR hal” gives you this post as the first result.

    I’d say I’m slightly above the next script kiddie, but thanks to folks doing what you’ve done, I’ve brought my MythTv box thru a lot problems to the semi-stable state it is.

    • Hello Matt,

      In fairness, the situation has gotten *much* better over the years. I built my first MythTV box in early 2006, and it took me two weeks. Given the time I’m sure you spent reading the above story, I’ll spare you the long tale other than to say it required me to recompile MythTV from source and break out a disassembler in order to identify what turned out to be that my video card didn’t support XVideo.

      Documentation does help, which is one reason I spend so much time working on the LinuxTV wiki (both adding content and vetting/maintaining the existing content). However, in most cases the *best* approach is to have the endurance to “not take the shortcut” and to work through the problem so that it can be fully understood and fixed. In the above anecdote, I had a variety of opportunities to “just make it work”. I could have gone with a different tuner card, I could have left the house and picked up a standalone $15 MCE remote control and receiver, I could have switched to using LIRC even though I knew the inputdev method was supposed to work “out of the box”. In the end, I thought it was worthwhile to work through the issue.

      The story is a reminder to me how bad the usability of remote controls is under Linux. From driver issues and poor logging to the nightmare that is getting lirc working, to getting the remote control profile properly defined for MythTV. It does have me thinking about ways that the process can be improved so that users get a better “out of the box experience”, even if it means optimizing such that the remote control that comes with the tuner “just works” while using some custom remote is a bit tougher. It’s a cost that I’m fairly sure every MythTV user goes through trying to get up and running, and assuming that they succeed eventually, rarely give a second thought afterwards.

      Devin

  3. Devin,

    I have done a new MythTV setup for my HDTV setup. I have my HDHomerun working great, but I cannot get any channels via the HVR-950Q. I am running Ubuntu9.10 (Karmic) from Oct 16. When I try to scan for channels with the 950Q I don’t get any signal status. Do you have any help on how I can get the HVR-950Q working?

    Thanks,
    –Ryan

    • Hello Ryan,

      You cannot really rely on the “signal status” indicator to provide reliable information as to whether the card is working – it will always show 0% (as is the case for most ATSC/ClearQAM tuners). You can try doing a scan from the command line to ensure that the card is working properly.

      Are you trying to scan ATSC? Analog? ClearQAM? Do you have the firmware loaded? Are there any error messages? Did you add the “no_poweroff=1” to the modprobe options for the xc5000 driver?

      Devin

  4. I have tried ClearQAM and ATSC HDTV OTA.

    I have the /etc/modprobe.d/xc5000.conf with ‘options xc5000 no_poweroff=1’.

    I am not sure which ‘scan’ from the command line you are refering to. I did ‘scan -a 0 -f 0 -d 0 -A 2 -c -5’ and here is the output:

    using ‘/dev/dvb/adapter0/frontend0’ and ‘/dev/dvb/adapter0/demux0’
    WARNING: filter timeout pid 0x0000
    WARNING: filter timeout pid 0x1ffb
    dumping lists (0 services)
    Done.

    Looking in /var/log/dmesg I didn’t see any errors. I am not sure what I am doing wrong, or what I am doing right.

    Thanks,
    –Ryan

    • Hello Ryan,

      You never specified the name of the scan file to use as an argument. Try:

      /usr/bin/scan /usr/share/dvb/atsc/us-ATSC-center-frequencies-8VSB

      The location of the file “us-ATSC-center-frequencies-8VSB” may vary based on your distribution.

      Devin

Leave a Reply