Skip to content


PCTV 74e and More Em28xx Quality Work

I spent the last few days working on some basic quality problems with the KWorld 2800d (an em2860/saa7113 design). It would seem that I uncovered a bug that was introduced almost four years ago which would have resulted in reduced quality for all saa7113 based products. I’m also working on a couple of formatting bugs on the em28xx.

I took a bit of time this evening and started looking at the code that Abilis provided for the as102. The chip in question is used in the PCTV 74e (otherwise known as the picoStick). PCTV Systems pushed for the code’s release under the GPL, and they deserve the credit for yet again helping out the Linux community (something I hope readers will remember when considering their next tuner purchase).

The firmware is also being released under terms that allow it to be freely redistributed with the Linux distributions, which means once we get this upstream then users will have a completely plug-and-play experience.

The code is going to need some cleanup before it will be accepted upstream, as well as testing. I’ll also be putting out a “call for testers” so users can provide feedback before it goes upstream. So if you own one of these devices and are looking for Linux support, stay tuned!

Posted in 74e, as102, em28xx, pctv systems.


More fun with lirc and remote controls

For those of you who were following my MythTV saga from a few posts ago, I am happy to report that as of tonight, all the buttons on my remote control now work from within MythTV. How much work required though really offers some more insight as to how badly broken usability in Linux is.

Originally, I thought I would be able to take advantage of the fact that the kernel knows what remote control ships with the device and is automatically generating input events by default (assuming you remove the cx88-ir driver from the hald blacklist). So for example, when I hit the “Channel Up” button on the remote, a KEY_CHANNELUP input event was being generated. Unfortunately, MythTV doesn’t understand any of these input events that are Linux specific. Myth relies on a table of input events provided by the QT library, which provides a common portability layer (at the cost of only offering the lowest common denominator). Hence, even though my remote control was detected by the kernel, and unique events were being generated for each key, there was no way for MythTV to identify the events.

I could probably make it work if I were willing to extend the MythTV source to know how to handle non-standard events (and I might still do that in the future). If such work were done, MythTV would start to “just work” with all of the remotes that are shipped with tuner products by default.

So I ditched that idea and figured I would go with the lirc approach. When I had tried running “gnome-configure-lirc” previously, it had failed to show my cx88 card in the list. But now that I had removed it from the hald blacklist, now it showed up properly. I selected the cx88 and specified the “Hauppauge silver/black remote” I had kicking around.

I saved the configuration and exited the program, but at that point the lircd service would not start, claiming that it could not load the kernel modules. It turns up the default lircd configuration file specifies two single quotes (”) as the name of the modules to load, and says that loading of the lirc kernel module is required, even though my device uses the inputdev interface. Edit the configuration file to say that no modules need to be loaded, retry, and find that now it’s complaining that it cannot open device profile (”), since the gnome-configure-lirc also doesn’t bother pointing lirc at the remote control configuration I had just created. More edits to the configuration, and now the lircd daemon *finally* starts.

I run the “irw” test utility and start whacking buttons on the remote, and it is showing the correct info for *most* of the buttons. However, several buttons fail to register at all, like the “OK” and “Back/Exit” button, both of which are fairly important. So now I’m running “irrecord”, a handy little command line application for mapping buttons to keys that, from the crappy usability, you would think was written by a twelve-year-old. You’re expected to know the exact names of the input mappings in advance, or else you have to run “irrecord –list-namespace” in a separate window to dump out the list. Oh, and it doesn’t do incremental updates properly so if you run the program twice it proceeds to trash out whatever keys you defined previously (forcing you to output to a temp file and merge the results by hand with a text editor).

It’s worth noting that the remote control profile setup by gnome-configure-lirc *does* have mappings for the buttons in question – they just don’t match what is actually being generated by my remote control for whatever reason.

I get past *that* point and now irw is showing the correct buttons, but MythTV doesn’t work still. It turns up at some point “mythbuntu-lirc-generator” got run, creating a profile based on a different remote control. It’s not quite clear to me what sort of black magic this program does to setup MythTV, but after digging through the frontend logfile it became clear that I needed to run it again.

In total, I spent about 90 minutes fumbling around before I finally got it all to work. There is clearly considerable room for improvement in this process. In fact, one might argue that it’s pretty pathetic how much work was required given it’s an extremely popular card and I couldn’t even get it to work with the stock remote that shipped with it.

At least my fiance can now finally use the MythTV box in my living room without having to know the esoteric commands like how the left and right bracket [] keys on the keyboard control the volume.

Posted in Uncategorized.


Status Updates…

I know things have been pretty quiet on the blog lately, so I wanted to give you all a quick status update on how various projects are going (for better or for worse). We are quite busy, but alot of the stuff we are working on is not currently publicly visible, and many people who read the blog are interested in various things we are doing in the open source community (either out of academic curiosity or because you own some piece of hardware that you are hoping to see better support for).

So, let’s talk about the stuff that we are actively working on first:

em28xx quality fixes – I spent most of this week trying to nail down a significant quality problem in em28xx analog support. I did isolate one problem, but I am seeing other issues that bother me. The current fix is here:

http://kernellabs.com/hg/~dheitmueller/em28xx-test/

While I was at it, I nailed down a regression that got introduced that resulted in the xc3028 firmware taking nearly five seconds to load (whereas it used to take about 1 second). This is probably most noticeable by users of devices such as the original HVR-950 (em28xx/xc3028) where it resulted in applications like tvtime and MythTV taking *much* longer to startup.

http://kernellabs.com/hg/~dheitmueller/xc3028-regression/

tvtime – Tvtime work has been slower going than I would have liked. I did do some work last week, but the more testing I do the more problems I find. The goal at this point continues to be to get a release where both the audio and VBI support work well with a variety of cards, and then ask the distributions to redistribute this version.

HVR-1600 ALSA support – There were a bunch of cleanups required to get this upstream, but I am happy to announce that it was merged this week. None of the changes asked for were actually necessary for the driver to work – all of them were codingstyle issues.

http://kernellabs.com/hg/~dheitmueller/hvr-1600-alsa-3/

Now, let’s talk about the things I’m not working on right now. This list is based largely on the comments I receive both on the blog and via email where people ask “When is X going to start working?”. I feel there is some value in telling everybody about this stuff, so that expectations are set properly and in case anyone else wants to contribute (knowing that we are not actively working on it).

HVR-950Q work with MythTV – Ok, so I was the guy who wrote the analog support, and I did do some work a couple of months ago to at least getting it basically working with MythTV (with a few well-documented caveats). But as at least one user has pointed out, there is some sort of race condition where the card gets locked up periodically when switching back and forth between analog and digital. If you use it for digital only then you are fine, or if you use it for analog only then you are fine. But if you try to use it for both then you are in for trouble.

HVR-900R2 and PCTV 330e – I’ve discussed this particular issue various times in previous posts. Basically it boils down to getting an extraction script written for the firmware, and fixing the existing driver to work with the updated firmware (since the firmware that was used to write the original driver can not be found). You can read the blog history for the gory details as to why something that sounds as simple as a firmware extraction script would end up being such a headache.

PCTV 340e/340eSE/801e/801eSE analog support – At this point, I have no plans to be extending the dvb-usb framework to support analog, which is what would be required to get analog support working for any of the dib0700 based devices named above.

saa7164 analog support – This is basically the same situation as with the dib0700 – it requires a *huge* development effort that would amount to hundreds of hours of labor. Sure, we might come out with something at some point if a commercial backer steps up, but I would not expect to see this supported in the immediate future.

I realize that the above list of “stuff we are not working on” can be frustrating if you are a user who was hoping to see one of these things done sooner. Let’s not forget that all of these are projects that KernelLabs developers are doing in their own free time for no compensation. We’ve donated thousands of hours to making LinuxTV better. And when it comes to the stuff we do in our free time, we need to focus on the things that interest *us* personally in order to keep our sanity. Unfortunately, the stuff I may be interested in “playing around” with is often not the stuff certain groups of users would prefer we spend our time on.

It’s probably also worth noting that we do see people who post comments in the blog along the lines of “How about accepting donations? I would donate $10.00 to see XYZ working.” However, at this time we are not accepting donations toward any of these projects. Why? Because there is *nowhere* near enough people who are willing to donate to make it worth our while – and it wouldn’t be fair to accept your donations but not deliver. I do appreciate those people who offer to donate, but unless you can find 1500 more friends like you, you are in an extreme minority. I have had extended conversations with a few individuals who argue that our view is not valid, but ultimately it is our decision what we dedicate our efforts to.

To end this post on a positive note, if you are a PCTV 74e user who has been looking for driver support, keep an eye on the blog since I’ve got a GPL driver and have worked out the firmware redistribution rights with the chipset vendor, so I just need to do some cleanup and testing before we can post a tree for testing.

Posted in Uncategorized.


Em28xx video quality

I’m sure you have all seen the following on a channel in the middle of the night. It’s a set of colorbars.

The above colorbars were taken using KernelLabs’ analog generator, in conjunction with an em28xx based device. I had been bothered about the general picture quality on the em28xx for some time, but without a good reference signal it can be very difficult to ascertain what is wrong with a picture.

If you click on the above image to see the full version, something might jump out at you. Notice the pixelation along the vertical edges of the color bars. This is an artifact of interlacing that is usually only seen when there is fast horizontal motion. But this is a steady picture, so we shouldn’t be seeing that here. So, why do we see this here?

I used the v4l2 capture-example tool to capture a raw yuyv sample, and then converted it to RGB and opened it in the Gimp. This approach works around the fact that the screenshot feature in tvtime saves in PNG format, and I don’t want to be confused by compression artifacts.

Once you have the raw frame opened in the Gimp, you can now take a *very* good look at it. And zooming into the top left corner of the frame shows the problem (magnified to 1600x):

There is an apparent problem with the formatting of the video such that the start of the top field is off by one pixel (the pixel is green because that’s the default color before the driver populates the buffer). This means that the two fields are misaligned, resulting in a zigzag pattern on vertical edges. And combined with the deinterlacing algorithms employed by application such as tvtime, it results in a slightly blurry image.

I’ll have to dig into the driver source and identify where the buffers are not being properly populated, but at least I know what the problem is. Looking at live video, all you can say is that the picture “just looks bad”, but with the proper test signals you can actually identify what is wrong with the picture so that you know where to look in the driver code.

Posted in Tutorial, em28xx.


Building a MythTV box

I want to share an experience with you all, since everybody loves to watch a train wreck. As developers, we like to think that the state of the LinuxTV project is continuously improving. Then I have days like yesterday….

I wanted to build myself a MythTV box for personal use. Since my goal is to use this “in production”, I figured I would pick a PCI based tuner that had mature driver support. I also assumed that this would minimize the likelihood of me running into trouble. I ended up choosing the PCTV 800i, which is a cx88 card that uses the xc5000 tuner and s5h1409 demodulator. It also has an onboard IR receiver, something which for once I actually cared about since I planned on putting this in my living room. It’s also worth mentioning that the cx88 driver provides RC events via the input layer for the remote control that ships with the PCTV 800i’s, it should “just work” without having to muck with lirc.

I built a box with a fresh install of Ubuntu 9.10, and the card was detected “out-of-the-box”. No digging around the Internet for firmware was required since everything was already included in the distro (including the updated xc5000 firmware KernelLabs got merged last year). I ran Synaptic and installed the MythTV packages, ran MythTV setup, and in a matter of minutes was watching an ATSC stream. Mythtv-setup found the card, the channel scanner worked as expected, and I was happy. The only thing remaining was to get the remote control working – and that’s when everything went downhill.

People often approach me about a problem with their tuner, and my first advice is always “update to the latest code and see if it’s still a problem”. Hence it was only natural that I should take my own advice. I pulled the latest v4l-dvb code from linuxtv.org, rebooted, and my PC hung on startup. I tried various boot combinations (such as single-user mode), and finally resorted to cracking open the case and removing the card in order to get the PC to boot. At that point, I was able to see the /var/log/messages from the failed boot attempts, which included an kernel OOPS in the IR initialization.

It’s probably worth noting at this point that I have been working on various projects for the last few weeks that all involved branches, so I had not running the v4l-dvb trunk for almost a month. So, as a test, I updated to the trunk on my laptop and plugged in my HVR-950 – and hit the same OOPS.

Somehow a regression was introduced into the trunk which causes an OOPS on both cx88 and em28xx based devices whenever the driver is loaded. The LinuxTV maintainer apparently has decided to use the v4l-dvb trunk as his personal sandbox, instead of putting his experimental changes on a branch where others can test them before they get merged into the mainline. To add insult to injury, this problem was reported by five different people with five different products on the mailing list over the last four weeks, and the one line patch still hasn’t been checked in.

Once I added the patch to my v4l-dvb tree, the driver started working again, but the IR support still didn’t work. I plugged in my HVR-950 and confirmed that the IR was working there, suggesting some difference between the two. I spent time comparing the way the IR subsystem was initialized by the two drivers, and didn’t see anything obvious. I then started added printk() calls to the IR code, and concluded that both devices were sending the same key sequence to the input subsystem, therefore the keys must be dropped somewhere in the input subsystem.

So now I have to download the full 2.6.31 kernel source so I can add debugging to the input subsystem and try to figure *why* the commands are being dropped. After several hours of littering the input subsystem with printk() calls, I have concluded that in fact the keys are being delivered to the keyboard driver, but not to the event interface because the input device handle is not in an open state (at which point I realize that if I drop out of X11 to the virtual console, the remote control works). More printk() statement and dump_stack() calls added to the input_device_open() calls show that on the em28xx device the event interface handle is opened by the hal daemon, but not in the case of the cx88 device.

So, now I’m downloading and starting to look at the source code and logs for the hal daemon. By looking at the logs, and correlating them against the source code (since the quality of the logging isn’t good enough to figure out what is going on *without* the source code), I determine that for some reason hald is not binding to the input device because the cx88 IR device is set to explicitly be ignored by hal. More digging through code reveals that there is a blacklist being read. Finally, I arrive at the following:

/usr/share/hal/fdi/preprobe/20thirdparty/lirc.fdi

<?xml version="1.0" encoding="UTF-8"?>

<deviceinfo version="0.2">
  <device>
     <match key="info.product" contains_ncase="saa7134 ir">
        <merge key="info.ignore" type="bool">true</merge>
     </match>
     <match key="info.product" contains_ncase="IR-Receiver">
        <merge key="info.ignore" type="bool">true</merge>
     </match>
     <match key="info.product" contains_ncase="cx88 IR">
        <merge key="info.ignore" type="bool">true</merge>
     </match>
     <match key="info.product" contains_ncase="bttv IR">
        <merge key="info.ignore" type="bool">true</merge>
     </match>
        <merge key="info.ignore" type="bool">true</merge>
     </match>
  </device>
</deviceinfo>

Yes, under Ubuntu, they have explicitly told hal to ignore cx88 based input device, but *not* to ignore em28xx based input devices, which is why the two behaved inconsistently. Presumably they added cx88 to the blacklist to avoid conflicts with lircd.

In summary, removing the one line from that blacklist file will result in the device being found by HAL and the remote control will start to work. But look at what I had to do to come to that realization? I’m a developer who has worked on LinuxTV 20-30 hours a week for the last two years, and it took me an entire day to make this work (after hours of digging through the source code to v4l-dvb, the Linux kernel, and hald). Imagine if I had been a regular user? I probably *never* would have gotten it working.

As developers, it is important to occasionally take a step back and look at the things we are doing poorly and how we can improve. It’s also important to eat our own dog food occasionally to get some perspective in what end-users actually experience attempting to use our software.

Posted in 800i, cx88.


New analog signal generator

Our new (previously owned) analog signal generator arrived tonight (a Promax GV-698/11):

Now, you might say “analog is dead” or “who cares about analog now that television has gone digital?” And the answer would seem to be “lots of people”. The reality is that as much as we like to talk about the advances in ATSC, ClearQAM, and the DVB variants, a large number of people out there still have analog and expect it to work. In the United States, the problem is especially evident since there are very few ways to access cable television without a settop box, meaning those people who do not wish to have an HD-PVR probably have a cable box connected to a tuner via an analog output.

Having a generator will make it easier to do testing of international standards (something that we do quite a bit of nowadays), and this generator in particular supports a variety of audio standards as well as teletext, meaning we can be confident that those functions will work even when we do not have access to a local signal source.

And finally, I personally have been quite dissatisfied with the general quality of a number of the different tuners and video decoders that I have worked with under Linux. Having a well calibrated signal source with a variety of patterns will allow me to improve the general picture quality for those devices. Whether that means making sure the registers for hue/saturation/contrast are properly programming, or being able to spot problems with formatting, resolution, or geometry.

It’s really easy to look at a TV picture and say “that doesn’t look right”. But figuring out what is actually wrong about the picture and what needs to be tweaked to fix it requires the right tools.

Posted in Uncategorized.


More tvtime work…

Tonight I finally finished the HVR-1600 ALSA cleanup and submitted what I hope is the final PULL request to get it into the mainline.

While I was there, I took a quick look at how well tvtime works with the HVR-1600 – in short: it doesn’t work, and it fails in a way that I suspect would leave most users scratching their heads (in fact, I was only able to understand why it didn’t work by examining the source code). I started to look at the issues in terms of what will be required to support the HVR-1600, and what would be required to make tvtime better report the reason for failure in cases where someone attempts to use an unsupported tuner.

The reason I’m mentioning this is because I am strongly considering dropping support for V4L1 in tvtime – only supporting V4L2. Why, you might ask? Because the code is littered with conditional logic depending on which version is being used, and any changes I make require twice as much testing to ensure that I am not causing a regression in V4L1 support. Plus, with a grand total of three obscure webcams that haven’t been converted to V4L2 (out of over a hundred products), it’s just not worth it to maintain the code.

By removing the V4L1 code, I can make things much cleaner, be more confident that the functionality that we *do* provide works, and I can modernize the interface to better identify things like device capabilities (which are currently absent because they weren’t available in V4L1). This means you will get sane error messages if you do things like try to use your PVR-150’s MPEG device in tvtime, or use a tuner that has a colorspace that isn’t supported in tvtime.

My goal at this point is to get tvtime working with as many modern tuners as possible, and if dropping support for a few really obscure tuners that people cared so little about that they were never even converted to V4L2 makes that goal easier, then I think that is an reasonable compromise.

Posted in hvr-1600, tvtime.


dib0700 IR fixes

In response to ongoing issues with the IR support for dib0700 and firmware 1.20, I spent most of yesterday refactoring the code.

When firmware 1.20 was introduced, the dib0700 switched from a polling model using a USB control message, to the messages being delivered on a USB bulk pipe. The code I originally added would do a blocking read on the pipe with a 50ms timeout. Because the dvb-usb-remote code makes use of the global workqueue, this resulted in the global workqueue being blocked 50% of the time. Also, the synchronous urb_bulk_msg() call would burn excess CPU time (reflected as an abnormal increase in the system’s load average when devices were connected).

These changes convert to using an asynchronous callback, which eliminates the expensive polling.

http://www.kernellabs.com/hg/~dheitmueller/dib0700_ir/

My hope is that as a side-effect, this also plays a little nicer with less-then-perfect USB host controllers, reducing the chance of issues with the Nova-T 500 (which people report as intermittent mt2060 i2c errors after several hours of operation when IR is enabled).

Posted in 340e, 801e, dib0700, hauppauge, nova-t-500, pctv systems.


PCTV 340e feedback

We received a good bit of feedback in the last 24 hours from users trying out the PCTV 340e driver, and I am happy to say that 100% of it has been positive. There were a few issues with new users not familiar with compiling v4l-dvb from source code, but everybody who has tried it so far has reported success.

That said, I want to take this opportunity to thank a few parties:

  • Lars Forseth, for loaning me a PCTV 340e Solo board for initial testing.
  • Our friends at Xceive for providing support on the xc4000, in addition to providing licensing to freely redistribute the firmware (we will be working on getting that bundled into Ubuntu and other distributions in the coming weeks).
  • PCTV Systems, and in particular Rainer Miethling for providing sample hardware as well as technical information about the PCB layout.
  • Davide Ferri, for providing some code attempting to port the xc5000 driver to work with the xc4000

It’s probably also worth pointing out that this project was not paid for by any of KernelLabs customers (and in fact, KernelLabs actually *spent* money to buy a DVB generator so I could complete the project). These are not trivial projects (this board took about 50 hours to get working). So if you find this work useful, feel free to show your support via Devin’s LinuxTV Support Fund.

Anyway, for those of you who were kind enough to do testing, please keep an eye on the blog over the next couple of weeks, since I will be doing some cleanup and it would be worthwhile for you to try the final tree *before* it goes into the v4l-dvb mainline.

Posted in 340e, pctv systems.


Hauppauge Nova-T 500 work…

In an effort to empty out my backlog of issues, I spent most of last night looking at the Hauppauge Nova-T 500 and the various issues people have reported with regards to the dib0700 1.20 firmware.

Now, there are some people who might argue that the breakage is my fault, since I was the developer who pulled in the version 1.20 firmware and made it work (required to address i2c bugs preventing the dib0700 from working with the xc5000 tuner chip). And those people would be correct in doing so. ;-)

Let’s talk about the issues…

The first issue is one related to rebooting. People have noticed that with the new firmware, if you reboot the box the card stops working. However, if you power off the computer and turn it back on, the card works. Some have claimed the behaviour as “seeing mt2060 i2c errors after rebooting”. I have reproduced the issue, and can confirm that in fact it has nothing to do with the mt2060. The entire dib0700 gets into a hung state, and the USB host controller halts the device, cutting off all communication with the bridge. The fact that people report this as “mt2060 errors” is because that is the first operation attempted when tuning, so those are errors seen in the dmesg output.

The other issue is what users have described as “mt2060 errors after several hours of use, which go away if I disable RC polling”. This is described in some detail on this Launchpad ticket, among other places:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/397696

I haven’t seen this firsthand yet, but I did spend some time looking at the IR changes that were required to make the 1.20 firmware work, and discovered something interesting: the IR polling doesn’t have it’s own thread, and uses the kernel’s global workqueue. As a result, the changes for 1.20 actually cause the global workqueue to be blocked for 50ms at a time, and then unblocked for 50ms, and so on. In other words, the global workqueue is in a completely blocked state 50% of the time, which could cause problems with whatever else in the system relies on the global workqueue (like say, the keyboard driver).

I actually discovered this because I thought perhaps the polling with a timeout on the bulk pipe was the problem, so I changed the timeout to zero (blocking indefinitely until data arrives), and as a result my keyboard stopped working.

I don’t yet know whether this is related to the mt2060 i2c errors, but it’s definitely a *bad* thing and should probably be corrected.

At this point, I will see if I can come up with a reproducible sequence to cause the mt2060 errors, and then see if moving the IR polling to its own thread has any effect. Unfortunately, because the errors are intermittent and supposedly only occur after several hours, this is likely to be a time consuming process.

Posted in dib0700, hauppauge, nova-t-500.