blog.kfish.org

CFFPP: linux.conf.au 2010

noreply@blogger.com (Conrad Parker) — Wed, 15 Jul 2009 03:53:00 +0000

The call for papers for linux.conf.au 2010 has been open for a few weeks, and closes soon (July 24).

I really want to encourage some talks about functional programming! The conference has a pretty strong developer focus, and most talks are about a practical topic. More importantly, we're looking for talks that inspire people to try new techniques, to approach design and troubleshooting with clarity and vigor (yarr!), to boldly consider that they should perhaps spend some time honing their craft before writing yet another application that inexplicably fails at runtime -- all in a friendly and entirely non-condescending environment of hackers having fun hacking.

Here's some suggestions for the kind of talks that I think could be interesting:

systems programming in Haskell/OCaml/whatever: how you wrote an interface to some hardware, handled lots of IO, controlled a robot, whatever
functional programming for kernel development: verification, security etc.
game programming: higher order design for 3D, AI etc.
proof vs. testing: (can anyone do a tutorial on proof without greek letters? not that Patryk Zadarnowski's talk about the Curry-Howard Isomorphism a few years ago wasn't *awesome*, but as a result of that people are clamoring (clamoring!) for some advice about how to prove their programs have no bugs).
some ... other ... practical benefit of functional programming!

The conference is in Wellington in January. January! it'll be windy, and it's in New Zealand!

Release: libfishsound 0.9.2

noreply@blogger.com (Conrad Parker) — Tue, 07 Apr 2009 22:48:00 +0000

Fishsound has moved to Xiph.org! The new home page is at http://www.xiph.org/fishsound/.

New in this release

This release contains security and other bugfixes:

Security fixes related to Mozilla bugs 468293, 480014, 480521, 481601.
Fix bounds checking of mode in Speex header
Handle allocation failures throughout due to out of memory
Added support for libFLAC 1.1.3
Add conditional support for speex_lib_get_mode() from libspeex 1.1.7 If available, this function is used in place of static mode definitions. For ticket:419
Check for Vorbis libs via pkgconfig, required for MacPorts etc.

A proposal for generalizing the byte-range referral HTTP Response header

noreply@blogger.com (Conrad Parker) — Tue, 07 Apr 2009 16:32:00 +0000

Re: the Media Fragments WD. Here I am using the term "byte-range referral" for multiple concatenated HTTP requests, for the purpose of improving cacheability; this is called a "4-way handshake" in the current working draft.

Shortcomings of the existing byte-range referral scheme

The above WD, and the current Annodex scheme, are specified to allow sharing of non-header data between different temporal views of media resources. They limit the positioning of custom data to the media headers. different segments to have different headers, which is useful for Ogg but not necessarily so for other formats.

Even for Ogg, it could be useful to refer to the codebooks separately from the Skeleton for more finely-grained data re-use. Then a client can locally cache the codebooks and know not to bother retrieving them over and over; but to get the updated skeleton and keyframe data for temporal segment requests.

Hence, I am proposing that we should specify an ordered list of tuples of (URI, byte range) which the concatenation of is byte-wise identical to the byte contents of the requested URI

This response can also contain data, so if you want to refer to this response you can include a tuple of (this, range) where this is the literal string "this", and refers to the body of the current response.

This syntax then allows the server to include parts from many different URLs. The custom data is then centralized in this response, and can be used for any parts of construction of the response so that it can be used for tail data (such as ID3 tags, divx seek tables etc.)

List and tuple separator characters

The list separator should be commas, as this then allows the list to be separated over HTTP response lines (without re-ordering).

Hence the tuple separator should not be commas; it can simply be whitespace:

Range-Referral: http://www.example.com/video.ogv?headers 0-1280
Range-Referral: http://content1.example.com/video.ogv 5380-48204
Range-Referral: this 0-950
Range-Referral: http://content1.example.com/video.ogv 60880-238382

By comma replacement, this set of headers is equivalent to the single header:

Range-Referral: http://www.example.com/video.ogv?headers 0-1280, http://content1.example.com/video.ogv
5380-48204, this 0-950, http://content1.example.com/video.ogv 60880-238382

Interpretation of other response headers

The body of this request is simply all the custom parts for this view, concatenated bytewise. The Range-Referral header explains how to use this data.

Content-Length: is the length of the body.

A Range request is made relative to the body. So for example a client could just do a HEAD request to get the Range-Referral headers, and then do multiple Range requests to retrieve the reqired parts in sequence (rather than locally caching all the data for tailers etc.). Coherence of the concatenated responses can be assured by the use of existing HTTP/1.1 caching identifiers.

So, this constructed response is only special in that a user agent knows how to use it in conjuction with other URI response data to display a media segment. Otherwise it is standard HTTP, and can have caching headers/tags attached, be cached by intermediate proxies, and itself be the subject of range requests.

Generalization to other segment types

This mechanism allows a complex sequence of byte-ranges to be specified. It explicitly marks data ranges which are re-usable, allowing them to be cached. It generalizes so that any complex data subview can be served, where re-usable data is keyed canonically and can be cached on the network.

For example, it may be useful for specifying the data for a spatial subrange of video.

liboggplay, liboggz, libfishsound migrated to git.xiph.org

noreply@blogger.com (Conrad Parker) — Fri, 03 Apr 2009 06:13:00 +0000

The source repositories for some Ogg libraries developed as part of the Annodex project have moved from from svn.annodex.net to git.xiph.org. These libraries are:

liboggplay, an Ogg Theora playback library used by Mozilla Firefox;
libfishsound, a simplified API for using audio codecs, used by liboggplay and the by the DirectShow Oggcodecs; and
liboggz, a library for seeking, reading and writing Ogg (used by liboggplay), and tools for managing Ogg streams. This includes oggz-chop, which is used by various sites including the Internet Archive to serve Ogg files.

Reasons for the migration

Xiph.org, which develops free codecs (Ogg Vorbis, Theora, Dirac, Speex, CELT, FLAC), already provided the hosting for Annodex.net projects. The move to the xiph.org domain reflects that these libraries are recommended for general use by projects requiring Ogg support.

The move from Subversion to Git allows for distributed development, letting developers without write access to the central Subversion repository develop code using a version control system, and making it easier for developers and packagers to track multiple independent changes. Among distributed version control systems, Git was chosen for its flexibility and popularity. It is already used within Xiph.org for Speex, the ultra-low latency, high quality audio codec CELT, and the experimental text overlay codec Kate.

Checking out the sources

To do a fresh checkout of the code, make a new git repository This assumes that you begin with an empty working directory:

$ git clone git://git.xiph.org/liboggz.git

Adding a remote to an existing git-svn checkout

Many developers already used git-svn to access the previous svn repositories. In this case you will already have a local git clone of the sources, perhaps with your own local changes. In that case, simply add a new remote to your existing repository, eg.:

$ git remote add xiph git://git.xiph.org/liboggz.git

Discovery and fallback for media segment addressing over HTTP

noreply@blogger.com (Conrad Parker) — Wed, 01 Apr 2009 06:56:00 +0000

This post concerns the use of queries or fragments in the URI specification for accessing segments of media over HTTP. We outline the user-visible differences between the two approaches, including the form of the URIs seen by users in each scenario and the consequent user interface activity, and then explain the HTTP request and response mechanisms that result. The purpose of this analysis is to better understand the trade-offs in usability and the impact on network performance, with reference to existing implementations rather than hypothetical scenarios.

I will make the case that the user-visible differences between the two syntaxes are immaterial, and that a more important distinction is that they induce different protocols. I will also claim that the use of the fragment syntax introduces unnecessary complexity in that it lacks a discovery mechanism and has no useful fallback to existing HTTP.

User-visible differences

We are constructing a URI syntax for addressing segments of media data. Taking the simple case of addressing some video content beginning at an offset of 10 seconds, we consider the two forms:

Query syntax: http://www.example.com/media.ogv?t=10
Fragment syntax: http://www.example.com/media.ogv#t=10

For simplicity here we are using a shortened segment identifier t=10; I touched on the topic of segment identifiers in a recent article about pretty printing durations.

Regarding the direct HTTP semantics of these two forms, if the user is already viewing the specified media.ogv, the query syntax reloads the portion from 10 seconds as a new resource, whereas the fragment syntax modifies the view of the current resource.

Although developers are rightly wary of a page refresh due to the time required to render complex HTML, in practice no visible change occurs when reloading a video. The query syntax has been used to control video seeking in JavaScript (using the Java cortado video player plugin, or an earlier Oggplay plugin), and also natively in the current Firefox 3.5 implementation.

In any case, this distinction is only user-visible if the video is the top-level resource. In the common case of a web page that embeds a video, the user-visible resource is the HTML page. In this case, the mechanism for controlling video is under the control of the embedding web page via JavaScript.

For example, URIs to YouTube pages allow a time segment to be appended using a fragment syntax. However, this fragment is used by JavaScript to control the embedded Flash video player; the mechanism for then retrieving video data is then managed by the Flash player. Similarly, in HTML5 Ogg <video> implementations, a fragment identifier appended to the HTML page may be interpreted by JavaScript to control seeking in the <video> source using a non-fragment mechanism, like query syntax.

Differences in request mechanisms

Either way we introduce a new behaviour that user agents can use to retrieve media segments over HTTP.

When handling a media segment which is specified by a query, the user agent initiates a standard HTTP request. It connects to port 80 on the specified host, and uses the entire path, including the query specifer, in the GET request. The server then begins transferring the required data representing that segment of the media.

To retrieve the URI http://www.example.com/media.ogv?t=10:

GET /media.ogv?t=10 HTTP/1.1
Host: example.com

However the proposed request mechanism for handling a segment specified by a fragment is not standard HTTP. In conventional HTTP, a fragment specifier is stripped by the user agent and not sent to the server at all; rather, the server sends the requested response (representing the entire resource), and after retrieval, the user-agent uses the fragment specifier to select the view shown to the user.

A recently proposed behaviour for handling media segments involves placing the segment specifier into the Range HTTP Request header, with a new units of seconds.

To retrieve the URI http://www.example.com/media.ogv#t=10:

GET /media.ogv?t=10 HTTP/1.1
Host: example.com
Range: seconds=10-

Response mechanism: byte-range redirection

The byte-range redirection response mechanism involves identifying parts of the segment view which are byte-wise identical to the original resource, and specifying redirections to those.

How discovery works

A user-agent will only receive a byte-range redirection response if it has indicated that it is capable of interpreting that, by including an extra HTTP request header. For example, here using a media segment URL specified with a query parameter:

GET /media.ogv?t=10 HTTP/1.1
Host: example.com
X-Accept-Range-Redirect: bytes

If the server is capable of handling the byte-range redirection mechanism, it will do so and indicate that it has done so explicitly in its response headers.

Query syntax has a sensible fallback to standard HTTP

However if the extra request header is not present, the server will simply send an entire response corresponding to the requested segment. Similarly if the header is present but the server is not capable of this new mechanism, it will simply continue with a standard HTTP response. The client can tell if the response is a segment response or not by the presence of an acknowledging response header.

If either client or server does not understand the byte-range redirection protocol, the request falls back to standard HTTP and the required segment is correctly returned. The cost of this fallback, compared to the case where both client and server understand the new request/response headers, is a loss of cacheability for subsequent overlapping segment requests.

Fragment syntax has a high cost of failure

The mechanism involving the fragment specifier does not have a fallback to standard HTTP: if the client does not understand that it should add the Range header with newly defined units, then it will end up simply requesting the entire resource. Similarly, if the server does not understand the new header then it will simply respond with the entire resource. If the cost of failure is to download some number of hours of extra video, as it would be in the case of MetaVid's congress proceedings, that is a prohibitive cost.

Summary

The distinction is one of protocol mechanism
For the common case of video displayed in HTML, the distinction is not user-visible
The use of fragment specifiers do not have a fallback to standard HTTP
The cost of discovery failure for fragments is high (retrieval of entire resource)

Actions

To clarify within the Media Fragments WG how queries can be used effectively, for both considered user scenarios.
To consider how the byte-range redirection mechanism can be generalized for other segment specifiers, such as spatial regions.

The economics of Twitter spam

noreply@blogger.com (Conrad Parker) — Sun, 08 Mar 2009 23:59:00 +0000

Recently more and more people have reported that they are being followed by spammers on Twitter. It's easy to track this problem: just search for #spam. Being followed by a Twitter spammer isn't like being stalked by a murderer; actually in the current environment, these guys are a fairly benign parasite that can work in your favor. So let's look at the economics of Twitter spam.

The upside for spammers is the usual obvious SEO shite: you've got something useless to peddle (yourself, your scam, your illegitimate business selling poor copies of pretentious luxury goods, your legitimate business selling enhancement placebos to suckers); you spend your time trying to defile fine and upstanding web pages with links to your pathetic piece of virtual real estate; Twitter comes along and your primitive brain realizes it can post its links there. You follow people so that they get a notification in their email pointing to your Twitter feed. Maybe they read it, maybe they click the tinyurl-obscured link. You cream yourself if they choose to follow you, because then they'll get all your spam, and you'll look more legit by having actual followers (like, real people from outside your cluster of bots and morons).

Now, what's the upside for normal humans in being followed by these scum?

Knowledge is work, a means for putting food on the table; information is power, a means for taking food from others.

Following as many people as you can on Twitter is a useful way to stay in front of your game: you know what people are up to, you see trends evolve, you get notice of articles before they're syndicated, you watch news unfold in your little niche of the world. And of course, the more people that follow you, the further your own message spreads: how great you are, how you're beating the system, how your ~~pretentious~~ beautiful designs and products can uplift and empower.

So there's an incentive to increase both the number of people you follow and the number of people who follow you. The first is easy; you just find people and press their button. The second is more difficult: you need to say something worthwhile in your tweets. Sometimes, not always, people will reciprocate when you follow them -- (SEO tip here!:) it helps if your own tweets are interesting.

However, there is a 2000 following limit: you can't follow more than 2000 people until you have 2000 followers. So, if you want to expand your reach into the info-verse, every follower counts -- even those spambots. So, now, these guys have evolved a little symbiotic, parasitic relationship with their hosts (you). You feel the first bite when they follow, but it feeds your ego. All you need is followers! no-one's going to do background checks on your popularity!

Relevance ranking anyone?

There's more to it though: Twitter search is currently being rolled out across the default user interface, and various bloggers are describing Twitter as a "search engine" (apparently that's the appropriate noun to describe someone that collects ideas). Twitter search is currently a realtime feed of query matches (the zeigeist! *fap* *fap* *fap*) with no relevance ranking. As the search feature gains usage, people will want relevant results to more complex queries. An obviously useful ranking input is the number of followers that a Twit has. These spambots will make you appear relevant!

We can follow this down silly paths -- eg. the more you tweet, the more spambot-followers you get, the more ranking relevance you have. The spammers introduce an incentive to posting often, and that mechanism has positive feedback.

More useful ranking mechanisms are things like reply frequency and analysis of re-tweets. Re-tweets are interesting to track because you can find the users who originate popular ideas: give them the microphone, dammit.

Action items

So there's an imbalance in the Twitter economy. Spammers are using Twitter and the environment encourages it.

Wishlist for Twitter:

Track how often users are blocked, warn against and auto-ban them.
Add user-initiated "Report spammer" buttons.
Implement detection of spammer clusters and auto-ban them.

Action items for Twitter users:

Block spammers on Twitter.
Block spammers on Twitter.
Block spammers on Twitter.

Please rant about how much you love the symbiotic parasitic relationship with your spambot-followers!

Random code: Pretty printing durations in Haskell

noreply@blogger.com (Conrad Parker) — Sun, 01 Mar 2009 03:13:00 +0000

Recently I've really enjoyed reading blog posts which just explain a little bit of code, so that's what this is. I had this code lying around from a few months ago so I added some context and links. It combines two of my favourite things: Annodex and Haskell!

YouTube's video offset syntax

Some time last year, YouTube introduced a feature which allows you to specify a hyperlink that plays a video from a given time offset. If you used the syntax on a random video site, it would look like this:

http://www.example.com/player.html#t=3m54s

That syntax for this is very close to that which we use in Annodex for Temporal URIs, now running on Archive.org (and soon on Wikipedia):

http://www.example.com/video.ogv?t=3:54

Two differences:

1. YouTube uses a fragment instead of a query parameter.

A fragment is something starting with '#' that tells the client to jump to a particular offset in the document -- in general the fragment text is never seen by the server. In the case of YouTube the HTML page contains JavaScript that tells the embedded Flash video player to seek to the offset in the video.

Fragments are useful in this use case, where you are instructing the embedding web page to play the video from a given time offset. How it actually retrieves the video from the network is not specified, but importantly there is no requirement for the embedding web page to be reloaded.

(This distinction between fragments and queries is part of the W3 Media Fragments WG discussion on syntax).

2. The syntax uses unit markers h, m, s to separate the parts of time, whereas our specification uses the kind of specifiers common in industrial equipment (and clock radios).

Perhaps one advantage of the format YouTube have chosen is readability: sometimes it is difficult to read times such as 03:36:14.

http://www.example.com/video.ogv?t=3:54
http://www.example.com/video.ogv?t=00:03:54.000
http://www.example.com/video.ogv?t=npt:00:03:54.000
http://www.example.com/video.ogv?t=smpte-25:00:03:54::0

We had a recent discussion about these issues in the Media Fragments WG: Action-28: updated syntax document with time formats. I'm pretty happy with the syntax we have settled on, allowing for both readable short timestamps and more accurate long ones.

Pretty printing of durations

Anyway, I was bored so I hacked up a sweet fold to display the format used by YouTube.

Haskell hackers use folds like C programmers use for loops; the Haskell wiki page Fold is a beautiful introduction to the topic. My favourite Web 1.0 interactive visualization of a left fold is at foldl.com (and also be sure to check out its companion site for right folds, foldr.com).

Here's a concise fold that gets us most of the way to the right syntax:

> ts = [("ms", 1000), ("s", 60), ("m", 60), ("d", 24), ("y", 365)]
>
> duz ms = ss
>   where (ss, _) = foldl (\(ss, x) (s, y) -> (show (rem x y) ++ s ++ ss, quot x y)) ("", ms) ts

Yeah, concise. Read it slow! if it was in C or Python, that one-liner would be a 10 or 5 line loop.

You might say that you use the fold function to iterate through a list of time units, and at each step of the iteration you do an integer division by the unit, label the remainder, and pass the quotient on to the next step of the iteration. A real Haskell programmer, however, might say something like "you fold the duration quotiently through the units, labelling into the syntax!", with much wringing of hands and wishful glances for abstract ponies. Fold is a verb, because functions are alive! Quotiently is not a word.

A problem with duz (apart from the crappy name) is that it shows times like 0y0h3m54s0ms. The next implementation of duration strips the leading and trailing zeroes:

> dur ms = years:rest
>   where (rest, years) = foldl (\(ds, x) y -> ((rem x y):ds, quot x y)) ([], ms) [1000, 60, 60, 24, 365]
>
> duration ms = concat $ map (\(n, s) -> show n ++ s) (takeWhile (not . zero) $ dropWhile zero labelled)
>   where labelled = zip (dur ms) ["y", "d", "h", "m", "s", "ms"]
>         zero (n, _) = (n==0)

eg. to display the duration of 2^32 milliseconds:

*Main> duration (2^32)
"49d17h2m47s296ms"

*Main> duration 3600000
"1h"

Fold is a generic list processing device; if you want to limit the amount of the list that is processed, you can use functions like takeWhile and dropWhile. These will take, or drop, elements from the list as long as some criterion is satisfied; you can use them both together to trim both the start and end of the list. Of course you can use these on the input list to limit what data is processed; but because Haskell evaluates lazily, you can also use these on the output list to limit how much of the processing is actually done (like in duration above). The bits of the evaluation that don't really need to get done, aren't: the idea of doing them is written down (on a "thunk") and thrown away. Burn your todo lists! Be lazy lazy lazy! Haskell rules. Do you like verbs?

Is OpenMAX important for Free Software?

noreply@blogger.com (Conrad Parker) — Tue, 24 Feb 2009 10:37:00 +0000

Much as OpenGL gives you access to 3D hardware, OpenMAX allows you to take advantage of hardware codecs. This is a brief overview introducing what OpenMAX is, explaining why it is useful for the open source community, and outlining steps for integration with free codecs, and open source multimedia frameworks and applications.

What is OpenMAX?

OpenMAX is a set of C APIs specified by the Khronos Group (who also co-ordinate standards like OpenGL and OpenAL). Whereas media frameworks like GStreamer and DirectShow are quite generic, providing all capabilities from codec integration through to synchronization of playback and recording and network access, OpenMAX more strictly defines three layers of operation:

OpenMAX IL (Integration Layer) is an interface to multimedia codecs implemented in hardware or software. It does not provide any interfaces for synchronized capture or playback of video and audio.
OpenMAX DL (Development Layer) APIs "specify audio, video and imaging functions that can be implemented and optimized on new CPUs, hardware engines, and DSPs and then used for a wide range of accelerated codec functionality such as MPEG-4, H.264, MP3, AAC and JPEG."
OpenMAX AL (Application Layer) provides acceleration of capture and presentation of audio, video, and images.

The significance of this layering is that it allows hardware and software developers to implement conformance to a particular layer, so that device manufacturers can more reliably integrate components from each. This creates a free market for media components as commodities; and of course open source businesses are well suited to operating in such an environment.

OpenMAX is already availabile in generally open source platforms like Maemo and Android. As part of my work with Renesas I've been developing OpenMAX IL components for the video encoding and decoding hardware on the SH-Mobile processor series. (However, this post does not necessarily reflect the views of my employer).

Open Source implementations

OpenMAX components implement a specific C API. All components need to manage their ports and synchronize access to their input and output data buffers, so implementations generally include a shared library for the IL core, as well as some OpenMAX components required to pass Khronos conformance tests. There are (at least) three open source implementations of OpenMAX IL:

Bellagio, developed mainly by STMicroelectronics and Nokia.
TI have an implementation of OpenMAX for OMAP.
OpenCore, the multimedia framework used by the Android platform, includes an open source implementation of OpenMAX IL. [gitweb]

So far I've been working with Bellagio, which has an active open source community. It has a good balance between commercial concerns like manufacturer deadlines and conformance testing, and openness to the community by encouraging and integrating development forks, and having a responsive mailing list and bug tracker.

Xiph OpenMAX

I haven't mentioned specific codecs yet; OpenMAX currently encourages use of non-free codecs like MP3, MPEG-4 and H.264. This in itself is not good for the aims of Free Software, but I think that the API standardization that OpenMAX offers can simplify the productization of hardware implementations of free codecs.

Xiph.org develops free codecs (Ogg Vorbis, Theora, Dirac, Speex, CELT, FLAC). Ogg Vorbis is required by the OpenMAX IL specification, but there are not yet any other OpenMAX IL implementations of the other codecs. Developing software OpenMAX IL components will allow application developers to implement Ogg support ahead of hardware support. It would also give hardware manufacturers a set of specific, well-defined goals for implementing Ogg support, with the understanding that the hardware components, when shipped with these software control APIs, will work in a variety of open source applications with minimal modifications.

There were a few Xiph.org people at FOMS 2009, so I introduced what we'd need to do to implement OpenMAX IL components for Xiph.org codecs:

Choose an OpenMAX IL framework
Implement generic Ogg mux/demux components (instead of single Ogg Vorbis component)
Implement IL components for each codec (Theora, Dirac, Speex, CELT, FLAC)
Implement GStreamer OpenMAX plugins for each codec

A recent thread, [Flac-dev] FLAC support for Android?, discusses requirements for implementing OpenMAX IL component for the lossless audio codec FLAC.

Free Software application support

In order to make use of OpenMAX components, applications need to either use the OpenMAX APIs directly or use a framework which does. For example, there is already an OpenMAX-GStreamer project which implements GStreamer plugin wrappers for Bellagio OpenMAX IL components. This allows any GStreamer application to take advantage of hardware codecs when they are available, or fall back to software implementations otherwise. This fits well with the GStreamer project's stated aim of of not implementing codecs, but providing routing, discovery and synchronization.

Other applications will need to use OpenMAX directly; good candidates would be applications that target mobile/embedded systems like Gnash, Fennec, WebKit and VoIP clients, as well as server-side transcoding or rendering software that needs high throughput.

Remember this:

Mobile processors increasingly have hardware units for video encoding and decoding, as well as audio and image processing
OpenMAX gives you access to hardware codecs (audio/video, image processing etc.)
Implementing OpenMAX components for free codecs will give manufacturers a clear path to hardware implementation

At some point in the near future it'd be great to get a few open source OpenMAX implementers together at a conference, ideally at a more general multimedia workshop like FOMS to discuss application integration. Perhaps at FOMS 2010, or FOMS Europe? In any case it'd be good to get some more discussion going: do you think OpenMAX is important for Open Source, and for Free Software? What other barriers do you think there are to hardware support for free codecs? And would you be interested in helping out with developing and testing OpenMAX support for your favourite codecs, and in your favourite applications?

A month of Mondays

noreply@blogger.com (Conrad Parker) — Sat, 21 Feb 2009 02:24:00 +0000

The last month or so has been fairly busy. I'll write more about each of these activities, but here's a quick summary of what I was up to (from about mid-January to mid-February):

Attended the Foundations of Open Media Software developer workshop, which was again held during the week before linux.conf.au in Hobart, Tasmania. I had helped organize the workshop (what little I could do from Japan), gave an update on the state of liboggz and an introduction to OpenMAX.
Participated in the Annodex Association AGM (and I got to announce the results of the election of the 2009 committee!)
Gave a talk at the LCA OpenMobile miniconf about my work with Renesas: developing software APIs for managing the hardware codecs of the SHMobile processor series, and OpenMAX IL support.
Chaired the LCA Multimedia miniconf, which I'd organized together with Silvia Pfeiffer
Gave a presentation at linux.conf.au, Ogg Chopping: Techniques for Programming Correctness and Efficiency, mainly about functional programming.
Was invited to join the W3C Media Fragments Working Group
Got confirmation of Mozilla's funding, contracted through Wikimedia, for work on Firefox's Ogg seeking and server negotiation. Michael Dale and I were interviewed by Rodney Gedda of Computerworld: Wikipedia video gets boost with $100,000 Mozilla grant.
Prepared and held my PhD intermediate thesis defence.

Somewhere in there I also managed to fit in a few days skiing in Hokkaido, and some time in the office in Tokyo. I've spent the last week relaxing back home in Kyoto and taking stock before getting back into things.

Break's over! onwards and upwards.

I love Mondays! Over the last two years I've learned a lot about how to get multiple tasks done in parallel -- a mix of GTD and some other techniques I've been developing. Unfortunately it sometimes means making sacrifices, like getting up in the wee hours of the morning to get work done on a ski trip; but every few months I also need a reset; I tend to go a bit crazy whenever I visit friends back in Sydney ;-)

If you've got any advice or words of encouragement please share them in the comments!

Tractorgen on github

noreply@blogger.com (Conrad Parker) — Tue, 23 Dec 2008 23:56:00 +0000

Tractorgen is now on github:

http://github.com/kfish/tractorgen/

REPOSITORIAL

The contents of this revision controlled document repository are a computer
source code implementation of TRACTORGEN, being a model of ASCII tractor
mechanics.

It is recommended that one study these documents closely in order to better
understand the finer details of the subject at hand. The authors firmly
believe that only through such preparation, preferably during the course of
one's daily study regimen, can a deeper appreciation of the theory be
attained.

As a side note, it has been noted by correspondents that it is possible to
derive a computer readable binary executable from these documents through
the use of sophisticated compiler technology. On the off chance that any
readers would wish to pursue this path, we include the apparent preparation
for doing so herein, as quoted:

$ automake -a
$ autoreconf

Upon completion of this procedure, which we expect should take on the
order of one to two weeks (of course the actual time depends on the
staffing resources of your local computer centre), a new document shall
be generated _as though from nought!_ [emphasis added]. The name of
this document is expected to be "configure", and it may itself be
executed thus:

$ ./configure

We recommend scheduling a vacation!

Upon your return, type "make", then "make install", and prepare your
experimental apparati forthwith:

$ tractorgen

Generates ASCII tractors.

Commit messages

One must eschew the typically terse and perfunctory style of commit messages that are common in software projects, and ensure that the purpose, significance, and experimental procedure for each incremental change are appropriately recorded.

Subscribe to the tractorgen commit feed

Obviously, commit messages are a good place to store source code for important tools: 9112c05.

         r-------
        _|
       / |_______\_    \\
      |          |o|----\\
      |_____________\_--_\\
     (O)_O_O_O_O_O_(O)    \\

Release: HOgg 0.4.1

noreply@blogger.com (Conrad Parker) — Tue, 23 Dec 2008 07:34:00 +0000

A new release of HOgg, on Hackage:

hogg-0.4.1

This contains updates to work with Hackage, the Haskell source package system; and also a new hogg man subcommand to generate man pages for subcommands.

Updated for Hackage

Hackage is Haskell's source packaging system. It makes it very easy to keep up to date with bleeding-edge releases.

You'll need the cabal command. This is already in Gentoo (emerge cabal) and Arch Linux (pacman -S cabal-install). If you're on a system where cabal is not already packaged, you'll first need to install GHC (eg. apt-get install ghc6 on Ubuntu 8.10 or Debian Lenny systems), then:

$ wget http://hackage.haskell.org/packages/archive/cabal-install/0.6.0/cabal-install-0.6.0.tar.gz
$ tar zxf cabal-install-0.6.0.tar.gz
$ cd cabal-install-0.6.0
$ chmod +x bootstrap.sh
$ ./bootstrap.sh

This will download and build the packages required to set up cabal. From there, a new Haskell package like hogg can be installed by simply doing:

$ cabal update
$ cabal install hogg

This will build and install hogg into $HOME/.cabal/bin (which of course you should add to your $PATH if you actually want to use anything you install via cabal :-)

man page output of self-documentation

hogg already generated its own help text, with runtime checking of example syntax. This release adds a hogg man subcommand which generates the same help text in Unix man page format:

$ hogg man man

.TH HOGG 1 "December 2008" "hogg" "Annodex"
.SH SYNOPSIS

.B hogg
.RI man
[
.I OPTIONS
]


.SH DESCRIPTION
Generate Unix man page for a specific subcommand (eg. "hogg man chop")

.SH OPTIONS
  -h, -?  --help     Display this help and exit
  -V      --version  Output version information and exit

.SH EXAMPLES
.PP
Generate a man page for the "hogg chop" subcommand:
.PP
.RS
\f(CWhogg man chop\fP
.RE
.SH AUTHORS

hogg was written by Conrad Parker

This manual page was autogenerated by
.B hogg man man.

Please report bugs to <ogg-dev@xiph.org>

Release: liboggz 0.9.8

noreply@blogger.com (Conrad Parker) — Fri, 04 Jul 2008 10:03:00 +0000

liboggz 0.9.8 includes the first release of oggz-chop, as well as support for the new karaoke codec OggKate.

oggz-chop can be used to serve time ranges of Ogg media over HTTP by any web server that supports CGI. The oggz-chop binary simply checks if it is being run as a CGI script by checking some environment variables, and if so acts based on the CGI query parameter t=, much like mod_annodex. It accepts all the time specifications that mod_annodex accepts (npt and various smpte framerates), and start and end times separated by a /.

All you need to do is set up the following Apache config:

ScriptAlias /oggz-chop /usr/bin/oggz-chop Action application/ogg /oggz-chop

, and all your Ogg files will be handled with oggz-chop, which means that you can put a time range on the end, like:

http://www.example.com/candidate_speech.ogv?t=00:23/00:26

The minimal amount of data required to play the section between 23 and 26 seconds will be sent to you, such that it plays back immediately from the time requested. As for caching, it generates Last-Modified HTTP headers, and responds correctly to If-Modified-Since conditional GET requests.

It implements the same chopping algorithm as the Haskell version hogg chop, released in HOgg 0.3.0, so it will insert an Ogg Skeleton track which can give players hints about what time the in-sync audio and video data should start being rendered, and if any of the input files include Skeleton information that will be preserved, and the output will contain only one Skeleton track.

Many thanks to Michael Dale, j^ and John Ferlito for testing out oggz-chop during its development.

FOMS 2009 CFP

noreply@blogger.com (Conrad Parker) — Mon, 30 Jun 2008 11:10:00 +0000

We recently opened the FOMS 2009 Call for Participation. FOMS — Foundations of Open Media Software — is a developer workshop "to widen cooperation and interoperability among open source media projects". It will be held a few days before linux.conf.au, in Hobart, Tasmania.

This year's FOMS had a large emphasis on free codecs, with many of the Xiph.Org developers in attendance. In 2009 we really hope to expand the participation to include people from projects with alternate technical viewpoints, such as those of MPlayer and NUT. It would also be good to get some exposure to projects like Canola2 and Bellagio, in order to deal with the issues of mobile multimedia. If you're involved in development of those or similar projects and would be interested in attending FOMS 2009, please respond to the CFP; there will be some travel grants available to help get you to Tasmania.

Related conferences

I'll be at Linux Plumbers Conference 2008 in Portland, Oregon, specifically for the "Audio" microconf.
We hope to run a Multimedia Miniconf at LCA 2009 :-)

:rikaichan for Vimperator

noreply@blogger.com (Conrad Parker) — Thu, 17 Apr 2008 14:45:00 +0000

Some of my favourite Firefox plugins are:

Rikaichan, a Japanese dictionary, which adds instant translation popups when you mouse over a word;
Vimperator, which provides vi-like user interface;
and Hide Tab Bar, because Vimperator's buffer list is more useful.

Vimperator hides the menu bar by default. Tools->Toggle Rikaichan has no default keybinding, and the keybindings to navigate the menubar are not available if the menubar is not visible, so Rikaichan can no longer be activated.

The following adds a vimperator command :rikaichan; save it to .vimperator/plugin/toggleRikaichan.js:

(function(){
    vimperator.commands.add(new vimperator.Command(
        ['rikaichan', 'rikai'],
        function(){
            rcxMain.inlineToggle();
        }
    ))
}) ();

It is aliased to :rikai for short, but unfortunately vimperator won't recognize :理解. Thanks to ktsukagoshi for the explanation of how to write a vimperator plugin (vimperatorのプラグインの作成).

Remember, the interface is inside your mind.

Continuation Fest 2008: Continuations for video decoding and scrubbing

noreply@blogger.com (Conrad Parker) — Mon, 14 Apr 2008 07:30:00 +0000

Yesterday was Continuation Fest 2008, at the University of Tokyo's campus in Akihabara (a very nice venue!). It was very well attended; latecomers overflowed to a second room and participated by video conference. It was a little strange to see so many people interested in such an ~~obscure, troublesome and malignant~~ expressively powerful programming construct; the breadth of talks made for a very inspiring and practical introduction to the theory, applications and implementation of continuations in many different languages.

I recommend reading Kenichi Asai's introduction to delimited continuations (slides [PDF]). He introduced the shift and reset operators through the problem of expressing exceptional control flow, and then explained how to use these to type (ie. determine a concrete type for) printf. The main point was that shift/reset provide a high-level abstraction over control flow, with minimal impact on the implementation of your existing functions.

Oleg Kiselyov demonstrated some new code for transactional web applications, using delimited continuations for explicit state sharing between parallel connections. The result is that the user has a consistent view across multiple tabs are open on the same site, and the state is transactional so that there is no need for warnings like "Do not press the BUY button more than once!". He said that everyone already understands delimited continuations, they just don't realize it.

The topic of my presentation at Continuation Fest was Continuations for video decoding and scrubbing:

Playback of encoded video involves scheduling the decoding of audio and video frames and synchronizing their playback. "Scrubbing" is the ability to quickly seek to and display an arbitrary frame, and is a common user interface requirement for a video editor. The implementation of playback and scrubbing is complicated by data dependencies in compressed video formats, which require setup and manipulation of decoder state.
We present the preliminary design of a continuation-based system for video decoding, reified as a cursor into a stream of decoded video frames. Frames are decoded lazily, and decoder states are recreated or restored when seeking. To reduce space requirements, a sequence of decoded frames can be replaced after use by the continuation which created them. ~~We outline implementations in Haskell and C.~~

Slides [383KB PDF]
Article [215KB PDF]

I'll be introducing the code for this over the next few months. Whereas in my presentation about video player internals at BOSSA I outlined the problem space in designing a multimedia architecture, at Continuation Fest I tried to break it down into subproblems and considered useful data structures and programming techniques for dealing with them.

I got a lot of great feedback, and I think I succeeded in my mission to introduce this problem space to some really smart people. Thanks particularly to Chung-chieh Shan for some insightful ideas about how to deal with existing stateful codec implementations. It was also very interesting to talk with Shinji Kono about Continuation-based C (cBc) (slides [HTML tarball]), a C-like language capable of expressing continuations, non-local jumps, multiple function entry-points, and assorted other ways to shoot yourself in the foot. He suggested that it was designed for exactly the kind of thing I'm doing, and I'll be interested to try it out. It is implemented in a modifed GCC 4.x as an RTL code generator, so should now be (fairly) architecture-independent.

Thanks to the organizers of Continuation Fest 2008 for putting together such a useful and interesting event. I look forward to implementing just some of the things I learned :-)

Release: Sweep 0.9.3

noreply@blogger.com (Conrad Parker) — Fri, 11 Apr 2008 21:43:00 +0000

This is a bugfix release of Sweep, addressing CVE-2008-1686. For details, see my earlier post about libfishsound 0.9.1. Thanks to Peter Shorthose for managing this release.

Release: libfishsound 0.9.1

noreply@blogger.com (Conrad Parker) — Mon, 07 Apr 2008 01:08:00 +0000

This is a maintenance release, fixing a security vulnerability in Speex header processing as outlined in oCERT 2008-02. When used in a client for web video content, as in the OggPlay Firefox Plugin or the Ogg DirectShow filters, a specially crafted Ogg Speex stream hosted on a server could be used to allow an attacker to execute arbitrary code on the client system. The OggPlay plugin binaries available from www.annodex.net have already been updated.

Details

The Speex header contains a 32-bit modeID field, interpreted by libspeex as a signed int (spx_int32_t) The normal way to use this is to index into a global mode list to retrieve a SpeexMode *:

mode = (SpeexMode *)speex_mode_list[modeID];

and then use that to set up a decoder:

st = speex_decoder_init(mode);

This calls speex_decoder_init() in libspeex, which looks like:

void *speex_decoder_init(const SpeexMode *mode)
{
   return mode->dec_init(mode);
}

So if you don't check that the modeID given in the stream header is within the bounds of speex_mode_list[], arbitrary code can be executed. libfishsound was checking the upper bound (modeID < SPEEX_NB_MODES) but was not checking against negative values.

Discussion

This header processing is all boilerplate, and a reference implementation is given in speexdec.c. I took a copy of that about 7 years ago for Sweep, which I then adapted for libfishsound. The current reference speexdec.c does not have this bug.

For the Symbian port of Speex we created a function which returns the desired mode given a modeID, rather than having application code index into a global mode list. I wrote and committed speex_get_mode() to libspeex in September 2004, and it does the correct bounds checking. So if I'd been using that function in libfishsound then today's problem would never have happened. As it turns out, the libfishsound svn trunk version of speex.c does use that function. As far as I am aware, the OggPlay plugin binaries have always been built against the libfishsound svn trunk, so they were never vulnerable in the first place. However, recent tarball releases of libfishsound have been coming of a separate branch, so the advisory is valid for applications linked against those releases.

Finally, I sent a patch to Jean-Marc Valin yesterday which entirely removes the possibility of this bug happening again by bounding the mode values returned by speex_packet_to_header() in libspeex. It will be available very soon in a libspeex release.

Acknowledgements

Thanks to the team at oCERT for the efficient reporting of this advisory, and to the anonymous submitter for the details. I was able to patch the offending branches, which allowed j^ to build and upload new OggPlay plugin binaries (within 24 hours of contact by oCERT).

Release: HOgg 0.4.0

noreply@blogger.com (Conrad Parker) — Mon, 24 Mar 2008 18:23:00 +0000

HOgg is a Haskell library and commandline tool for manipulating Ogg files. This release contains a bunch of code written during FOMS and LCA 2008, including a new sort subcommand and proper handling of Skeleton when merging and ripping files. Full details are in the release notes.

sort implementation

My favourite part is the implementation of the new sort subcommand:

sort :: [OggPage] -> [OggPage]
sort = sortHeaders . listMerge . demux

This is somewhat shorter than the equivalent C implementation, oggz-sort.c — Haskell affords abstraction whereas in C it's a trade-off. sortHeaders is a long (21 line) function that re-orders header pages according to the Theora and Skeleton specifications, and listMerge is a generic list merging function, also used in the merge subcommand. demux is tiny:

demux :: (Serialled a) => [a] -> [[a]]
demux = classify serialEq

You can read that as "demux is classification by serial number": classify is a generic list function, classifying list elements according to some criterion you give it. Here, for example, the list of pages:

[Video0, Audio0, Video1, Audio1, Audio2, Audio3, Video2, Audio4, Video3, ...]

will get classified into two separate lists:

[[Video0, Video1, Video2, Video3, ...],
 [Audio0, Audio1, Audio2, Audio3, Audio4, ...]]

This is done lazily, meaning that the processing is done on the fly and big intermediate lists are not constructed in memory. Video0, Audio0 will be passed through listMerge and sortHeaders and written to disk by the consumer of sort well before Video103 and Audio5007 are seen.

Documentation improvements and self-checking

The help for each subcommand now contains long descriptions, mostly similar to the man pages of the Oggz tools. The descriptions also have explicit sections describing how Theora, Skeleton and chained files are handled. The example commandlines for each subcommand use the Ogg MIME types and file extensions that we are now recommending in Xiph.Org.

The best bit though is hogg selfcheck, which checks that the help examples are valid. It checks that all the example commandlines pass through getOpt without errors, and that all file extensions used in options are valid. This is the kind of nice touch which would have been a pain to code up in C, but fell out cleanly in the Haskell implementation. As it is fairly cheap to run (and printing help text is hardly a performance-critical operation), this option is also silently run after printing out any help output at all, so that such errors are more likely to be found and reported. The same commit that introduced hogg selfcheck also fixed two such documentation errors which were found by this option :-)

Release: xsel 1.2.0

noreply@blogger.com (Conrad Parker) — Mon, 24 Mar 2008 18:17:00 +0000

XSel is a command-line tool for manipulating the X selection. This is a maintenance release, improving argument handling, documentation and X11 library detection.

BOSSA 2008: Video Player Internals

noreply@blogger.com (Conrad Parker) — Mon, 24 Mar 2008 17:29:00 +0000

Last week I attended BOSSA, a conference on open source software for mobile embedded platforms, organized by INdT. It was held in the town of Porto de Galinhas, Brazil. Since then I have been hanging out in the INdT labs in Recife, hacking on xine, catching up with friends and exploring the old city.

The topic of my presentation at BOSSA was Video Player Internals:

Embedded platforms put demands on latency and memory use. Video playback makes these difficult to guarantee. This presentation discusses the architecture of video players, and the problems imposed on them by the design of video codecs and their containers. To explain these problems we look at both proprietary and open source formats (MPEG, Ogg, Theora, Dirac, etc.) and evaluate open source video players in this context. We particularly examine xine and GStreamer, and introduce the minimal architecture of OggPlay.

Slides [613KB PDF]
Article [330KB PDF]

I'm very grateful to INdT for the opportunity to attend, it was an awesome conference in a very beautiful part of the world.

Release: liboggz 0.9.7

noreply@blogger.com (Conrad Parker) — Fri, 15 Feb 2008 08:54:00 +0000

There's been a whole bunch of work on liboggz recently; it deserves a few more weeks of shaking out and perhaps some updated Win32/MacOS support before it gets 1.0 slapped on it.

liboggz 0.9.7 includes a new tool called oggz-sort, which addresses a problem with some encoders that Shane Stephens brought up at FOMS. The discussion was going around in circles, so my response was to write this C code. It implements a function that Shane has written but not yet released in his OCaml implementation of Ogg (oogg), and which I've written but not yet released in my Haskell implementation (HOgg). Of course, people will take this version more seriously because it's written in C.

From oggz-sort (1):

oggz-sort sorts an Ogg file, interleaving pages in order of presentation time. It correctly interprets the granulepos timestamps of Ogg Vorbis, Speex, FLAC and Theora bitstreams, and all bitstreams of Annodex files.
Some encoders produce files with incorrect page ordering; for example, some audio and video pages may occur out of order. Although these files are usually playable, it can be difficult to accurately seek or scrub on them, increasing the likelihood of glitches during playback. Players may also need to use more memory in order to buffer the audio and video data for synchronized playback, which can be a problem when the files are viewed on low-memory devices.
The tool oggz-validate can be used to check the relative ordering of packets in a file. If out of order packets are reported, use oggz-sort to fix the problem.

This release also adds support for the experimental CELT audio codec, which is being developed by Jean-Marc Valin (the primary author of Speex). CELT is designed as a low-latency codec for high-quality audio. When wiretapping conversations encoded in CELT, we recommend that you record using the Ogg container format. You can then use oggz-tools to help with your analysis.

Release: xsel 1.1.0

noreply@blogger.com (Conrad Parker) — Sat, 09 Feb 2008 23:30:00 +0000

This is a story about the meaning of "version 1.0". A few weeks ago I released version 1.0 of xsel, a simple commandline utility for manipulating the X selection and clipboard. I chose to call it 1.0 after recalling a discussion with André Pang, about how the meaning of version numbers in open source software tends to differ from that in other software communities. For example, it is often advised not to buy the first version of a proprietary software product as it is sure to be buggy and incomplete; open source projects on the other hand often aspire to 1.0 being a major milestone, bug-free and fully-functional. The Windows and Mac freeware and shareware communities tend to follow a middle ground, content to release a useful but incomplete version 1.0, but thereafter avoiding the quick version creep that afflicts companies with marketing departments (and version-limited support contracts).

I'll argue that that middle way makes for more meaningful version numbers. Putting the label "1.0" on a release should be your way of saying that it's the first version that:

won't hose a user's system, and
hopefully does something useful.

Any version number less than 1.0 is sending out a signal that the software isn't quite ready yet; perhaps that you could lose or damage data by using it. Many people intuitively wait for version 1.0 before trying out some software, and this is fair enough. In fact, we need a way of warning that a project isn't ready for widespread adoption, that it could damage data, that the tarball is only out there so that other people can grab the code and help fix bugs. That's what version numbers less than 1.0 mean.

After 1.0, you can keep adding features and bumping the version number, working towards version 2.0 which perhaps does useful things in a different way. And from 2.0, onwards to 3.0 and beyond; integers are cheap. The important thing is not fall into the trap of thinking of 1.0 as some kind of asymptotic upper bound representing the perfect release.

Back to xsel. At first I wrote up release notes as version 0.9.7, but then then remembered that discussion with André and realized that it should really just be 1.0. More to the point the previous release (in July 2001, which went five years without a bug report or patch) should have been 1.0. So yeah, it was a good feeling to just write "1.0" and send it out.

The morning after releasing 1.0 I got a report from someone who couldn't get it to compile -- turns out they didn't have the X11 development libraries installed, and for some reason I had commented out the checks for that in configure.ac while testing something or other a while ago. As a result, the configure script wasn't check for its only dependency. I considered doing a canonical 1.0.1 (LOL) release. Within the next day, though, I got a report about how to fix handling of COMPOUND_TEXT, an archaic way of handling international text (since superceded by UTF8_STRING). And there follows the next lesson (as berated by Raster): random bug reports stream in after a release, not before. Erik de Castro Lopo is up to his 20th pre-release of libsndfile 1.0.18; each pre-release he gets bombarded with reports.

Anyway, the post-1.0 bug reports have died down, so today I'm releasing version 1.1.0 of xsel. "This release adds basic support for COMPOUND_TEXT and fixes a configuration bug". And I'm still waiting to hear good uses of xsel --append and xsel --follow.

FOMS, LCA Multimedia 2008: Videos

noreply@blogger.com (Conrad Parker) — Fri, 08 Feb 2008 05:58:00 +0000

I arrived back in Japan after a few awesome weeks in Australia for FOMS and LCA. The weather in Melbourne was great, and the food was fantastic.

Between FOMS and LCA, dozens of free multimedia software developers were in town. It was the first time that developers of Dirac, Speex, Theora, Vorbis, Ogg, and most of the Annodex crew were all in the same place, so we spent most of the week of LCA holed up in a room designing content description and packaging formats. One immediate outcome will be finalization of the Dirac mapping into the Ogg container.

I organised the multimedia miniconf on the Monday of LCA, which was jam-packed with excellent presentations and lightning talks. Thanks to everyone who came, and talked, and video recorded. There were plenty of comments along the lines of it being "pretty hardcore for a miniconf". If you are interested in helping with next year's LCA Multimedia, or have friends in Hobart who might be able to help, let's start throwing around ideas. In particular, quite a few people asked what happened to the audio miniconf parties from a few years ago, and it might be a good chance to revive those ...

Videos

The following pages contain embedded videos of the presentations from these events, and the multimedia-related presentations from LCA:

FOMS Proceedings: introductions by the participants
LCA Multimedia: Dirac, Xiph, EngageMedia, FFADO and many others
Multimedia talks @LCA: PulseAudio, Ogg, Theora, Telepathy, Farsight ...

The videos on these pages are embedded with mv_embed, which supports playback via the OggPlay plugin for Firefox, vlc-plugin or generic application/ogg. mv_embed is a JavaScript library by Michael Dale of MetaVid. It is really easy to use, you just include that library (<script src="...">) and then write <video src="..."> anywhere in your page. No need to wait for native HTML5 support in your browser :-)

Release: liboggz 0.9.6

noreply@blogger.com (Conrad Parker) — Sun, 13 Jan 2008 05:11:00 +0000

This release of Oggz 0.9.6 contains a new tool, oggz-comment, which can be used to edit the basic metadata (title, producer, copyright etc.) of Ogg Theora files. The library also has some pretty major improvements to the way it works out timestamps and does seeking, mostly the work of Shane Stephens.

In media files, timing and synchronization is extremely important. If the image and audio start to go out of sync, it is very noticeable and the video quickly becomes unwatchable. When you scan through a file you often need to decode a lot more data than you actually display. This is particularly the case when you jump backwards, which is common in a user interface that supports scrubbing. As video frames are stored as a difference relative to earlier (or later) frames, you end up needing to secretly jump further back in the file to the previous keyframe, and then decode many frames up to the one you actually want to show. For a smooth user experience you need to do this as quickly as possible.

Ogg has some interesting framing properties. Given that timing is so important, you might expect that every packet has its precise timing information associated with it. In Ogg, it turns out not to be so. Packets are stored in pages, and there is only one timestamp per page. It is common for many audio packets to be crammed onto one page; the timing information for all the rest is not stored in the file. On the other hand, the encoded data for video keyframes is usually much larger, and spans multiple pages. Only the last packet on a page has its timestamp recorded, so if the keyframe is followed by an a much smaller packet of frame data in the same page, the timestamp for the keyframe will be lost. For these reasons I tend to refer to Ogg as a "lossy" container.

In order to minimize these problems, liboggz now inspects the encoded data in order reconstruct the expected granulepos (corresponding to a timestamp) for every packet in an Ogg stream. This allows applications to use reliable timestamps, even though these are only sparsely recorded in most Ogg streams. This is not as easy as it sounds, particularly for Ogg Vorbis. To get a flavour of what's involved, read Shane's rant in the comments, explaining how to calculate Vorbis timestamps.

For an in-depth discussion, come to Ralph Giles' talk at linux.conf.au, Seeking is hard: Ogg design internals.

Release: xsel 1.0.0

noreply@blogger.com (Conrad Parker) — Sat, 12 Jan 2008 15:53:00 +0000

XSel is a command-line program for getting and setting the contents of the X selection. You can use xsel in shell scripts and desktop keybindings, so that the contents of the X selection are available to command arguments:

mozilla --remote "openurl(`xsel`)"

This release adds UTF-8 support and fixes various bugs. The last version of XSel was 0.9.6, released sometime around 2001. It may have been the first version also. For some reason a bunch of patches came in recently, and I've had the joy of revisiting this project.

For old time's sake, my thoughts on ICCCM. (Warning: explicit language). Back then I made a point of implementing as much of that crack as possible. You can even tell applications to delete their selected text:

To delete the contents of the selection: xsel --delete

(This really works, you can try it on xedit to remotely delete text in the editor window).

This time around, of course, nothing does what the docs say anymore. So we ignore the details in the 2001 proposal for Inter-Client Exchange of Unicode Text and just grunt atoms at the selection owner until they yield all their secrets. And now, finally, xsel works on Japanese.

People have come up with some interesting uses for xsel over the years, but nobody has yet come up with a nifty use for the following options:

To append to the X selection: xsel --append < file

To follow a growing file: xsel --follow < file

Any ideas?