Stefan Westerfeld: KDE2.1 multimedia planning

List:     kde-multimedia
Subject:  KDE2.1 multimedia planning
From:     Stefan Westerfeld stefan@space.twc.de
Date:     2000-10-28 21:10:33


I think as we have done for KDE2.0, we should again have a rough plan what
we want for 2.1 in the multimedia department. So here is something discuss.
Any comments appreciated.

   Cu... Stefan

 - snip -

KDE2.1 Multimedia Plan (draft)

KDE 2.0 was quite a step for multimedia on the desktop. Integration of aRts,
which took a year to do, provides an always running soundserver, and service
like media playing based on this. The notification system, backends and the
media player were completely redone, and users given a more powerful platform
for multimedia tasks.

Still, there remains a lot to be done. This document tries to provide a
planning and discussion base for coordinating "the way to go". The focus
lies on what can be achieved for KDE2.1.

Discussion and throwing in own ideas is highly appreciated at this point of
time. Also, please consider helping with the development.

This draft has spontaneously been designed on IRC, by

  Charles Samuels
  Stefan Schimanski
  Stefan Westerfeld


1. The KDE Media Player
2. New media types
3. Organization & Documentation
4. Midi/Sequencer
5. Not in 2.1

1. The KDE Media Player

1.1 Merging noatun and kaiman

Until now, kaiman and noatun were seperate media players, which were available
for KDE2.0 - kaiman was shipped with 2.0, noatun available via CVS. The plan
for 2.1 is to combine the result into one media player, which should then
be the official "KDE Media Player".

Technically, this bases on noatuns plugin architecture, and making kaiman
just one more way how noatun could look or behave.

1.2 Pluggable effects

aRts allows much more than vanilla playback. Filters can be used to affect
how things should sound. Currently, this feature has been mostly unavailable
in the media player(s), mostly for two reasons:

 1. there were not too many effects
 2. binding a GUI to effects was not-implemented-yet

The fix for (1) seems obvious: write more effects. One starting point could
be porting the FreeVerb code to the aRts architecture.

The fix for (2) is less obvious. The problem is that a way needs to be found
how GUI objects (which run in the player process) can be connected to the
effects (which runs in the sound server process). The clean way to do this
is extending MCOP to provide a signals & slots technology.

The idea is that you can for instance have a GUISlider aRts object, which
runs in the player process space, and a ExtraStereo plugin, which runs in
the artsd process space. Then you can say:


and thus cause a connection between the slider and the effect. On the other
hand the same mechanism can also be used to connect output GUI elements to

Besides replacing the need to do polling, this also allows GUIs being
redesigned very flexible, using the same modular approach that artsbuilder
provides now for sound.

1.3 Moving the tag reading code

The ID3 tag reading code, which provides information like title, author,...
from .mp3 files should be moved to aRts components (it's right now a noatun
plugin), to make it available to all applications.

2. New media types

2.1 mikmod

There is work going on on making an aRts PlayObject for mikmod files, which
would allow the KDE Media Player to play .xm, .s3m, .mod and others. As always,
the component would be available to all other apps who know how to talk to
aRts as well.

2.2 midi via timidity

A big idea would also be to make a timidity plugin for aRts. Timidity is a
package which renders midi notes to audio output using samples. Currently,
kmidi uses this to play its songs. Making an aRts PlayObject would again
provide the user with a much more consistent experience.

3. Organization & Documentation

3.1 KDE multimedia webpage

KDE multimedia development needs to be more visible. There is no official
coordination point where people can see whats going on right now. To solve
this, a KDE multimedia webpage (how does http://multimedia.kde.org sound?)
should be created. 

Charles has volunteered to take care of this.

3.2 aRts documentation

First of all, the multimedia chapter of the KDE2 development book should
become available to the public RSN. While this is a significant piece of
good developer documentation, this is probably not enough. More work
needs to go into a consistent documentation on http://www.arts-project.org
for both, developers and users.

3.3 kdelibs/arts -> arts

Packaging aRts seperately is probably a good idea. aRts itself doesn't
depend on qt/kde, and this is intended. People outside the KDE world should
still be able to use aRts as sound server or for music tasks.

If - for instance - Gnome would start using aRts, having it mixed in the
CVS and in packaging with KDE is probably a bad idea.

4. Midi/Sequencer

4.1 Usability issues

aRts in the CVS provides already midi realtime synthesis. You can do midisend,
and run instruments in artsbuilder. You can also combine it with Brahms or
artstracker to compose songs. The problem with it is, that if you are not
a rocket scientist, and study the code or collected READMEs for a while, you
will probably not manage to do it. That has to change.

A good start would be providing an instrument manager, where you can
graphically assign which midi channel should get which instruments, without
bothering about the details.

4.2 Interoperability

There are at least three sequencer-like programs which actively want to talk
to aRts:

 * Brahms - which has been in the CVS for some time now - Jan is working on a
            new version right now - some aRts->midi support is in right now
 * Anthem - nothing done right now, Pete definitely wants to have audio stuff
            like hd recording, and probably aRts->midi support should be
                        added, too
 * ArtsTracker - an experimental small tracker in kmusic

Especially the areas

 * connecting to aRts
 * finding (virtual) midi channels and synthesis instruments
 * assigning instruments to channels
 * sending timestamped events

need to be standarized more for midi. Other interesting fields include

 * harddisc recording
 * effect processing

Most of this only needs a small amount of code in aRts and the applications,
the problem is finding a sophisticated standard.

5. Not in 2.1

5.1 Media interfaces

aRts supports streams between objects. Besides the typed (float) audio streams,
it also supports untyped byte streams. The idea of extending these towards

 * which encoding does it have?
 * what is the size of the frames?
 * which streams (audio/video) belong together?

and others would make a good step towards further modularizing audio/video
codecs. It would also allow things like connecting from a decoder to a
renderer without knowing which conversions need to be done to make them
the same format (a connection manager could find a suitable chain of filters
to plug in there). Also http streaming and others could be supported.

Not in 2.1 however.
  -* Stefan Westerfeld, stefan@space.twc.de (PGP!), Hamburg/Germany
     KDE Developer, project infos at http://space.twc.de/~stefan/kde *-         
Kde-multimedia mailing list

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends, & analysis