List: kde-multimedia Subject: KDE2.1 multimedia planning From: Stefan Westerfeld stefan@space.twc.de Date: 2000-10-28 21:10:33 Hi! I think as we have done for KDE2.0, we should again have a rough plan what we want for 2.1 in the multimedia department. So here is something discuss. Any comments appreciated. Cu... Stefan - snip - KDE2.1 Multimedia Plan (draft) ============================== KDE 2.0 was quite a step for multimedia on the desktop. Integration of aRts, which took a year to do, provides an always running soundserver, and service like media playing based on this. The notification system, backends and the media player were completely redone, and users given a more powerful platform for multimedia tasks. Still, there remains a lot to be done. This document tries to provide a planning and discussion base for coordinating "the way to go". The focus lies on what can be achieved for KDE2.1. Discussion and throwing in own ideas is highly appreciated at this point of time. Also, please consider helping with the development. This draft has spontaneously been designed on IRC, by Charles Samuels Stefan Schimanski Stefan Westerfeld Overview 1. The KDE Media Player 2. New media types 3. Organization & Documentation 4. Midi/Sequencer 5. Not in 2.1 =============================================================================== 1. The KDE Media Player =============================================================================== 1.1 Merging noatun and kaiman Until now, kaiman and noatun were seperate media players, which were available for KDE2.0 - kaiman was shipped with 2.0, noatun available via CVS. The plan for 2.1 is to combine the result into one media player, which should then be the official "KDE Media Player". Technically, this bases on noatuns plugin architecture, and making kaiman just one more way how noatun could look or behave. 1.2 Pluggable effects aRts allows much more than vanilla playback. Filters can be used to affect how things should sound. Currently, this feature has been mostly unavailable in the media player(s), mostly for two reasons: 1. there were not too many effects 2. binding a GUI to effects was not-implemented-yet The fix for (1) seems obvious: write more effects. One starting point could be porting the FreeVerb code to the aRts architecture. The fix for (2) is less obvious. The problem is that a way needs to be found how GUI objects (which run in the player process) can be connected to the effects (which runs in the sound server process). The clean way to do this is extending MCOP to provide a signals & slots technology. The idea is that you can for instance have a GUISlider aRts object, which runs in the player process space, and a ExtraStereo plugin, which runs in the artsd process space. Then you can say: connect(slider,"value_changed",effect,"depth"); and thus cause a connection between the slider and the effect. On the other hand the same mechanism can also be used to connect output GUI elements to effects. Besides replacing the need to do polling, this also allows GUIs being redesigned very flexible, using the same modular approach that artsbuilder provides now for sound. 1.3 Moving the tag reading code The ID3 tag reading code, which provides information like title, author,... from .mp3 files should be moved to aRts components (it's right now a noatun plugin), to make it available to all applications. =============================================================================== 2. New media types =============================================================================== 2.1 mikmod There is work going on on making an aRts PlayObject for mikmod files, which would allow the KDE Media Player to play .xm, .s3m, .mod and others. As always, the component would be available to all other apps who know how to talk to aRts as well. 2.2 midi via timidity A big idea would also be to make a timidity plugin for aRts. Timidity is a package which renders midi notes to audio output using samples. Currently, kmidi uses this to play its songs. Making an aRts PlayObject would again provide the user with a much more consistent experience. =============================================================================== 3. Organization & Documentation =============================================================================== 3.1 KDE multimedia webpage KDE multimedia development needs to be more visible. There is no official coordination point where people can see whats going on right now. To solve this, a KDE multimedia webpage (how does http://multimedia.kde.org sound?) should be created. Charles has volunteered to take care of this. 3.2 aRts documentation First of all, the multimedia chapter of the KDE2 development book should become available to the public RSN. While this is a significant piece of good developer documentation, this is probably not enough. More work needs to go into a consistent documentation on http://www.arts-project.org for both, developers and users. 3.3 kdelibs/arts -> arts Packaging aRts seperately is probably a good idea. aRts itself doesn't depend on qt/kde, and this is intended. People outside the KDE world should still be able to use aRts as sound server or for music tasks. If - for instance - Gnome would start using aRts, having it mixed in the CVS and in packaging with KDE is probably a bad idea. =============================================================================== 4. Midi/Sequencer =============================================================================== 4.1 Usability issues aRts in the CVS provides already midi realtime synthesis. You can do midisend, and run instruments in artsbuilder. You can also combine it with Brahms or artstracker to compose songs. The problem with it is, that if you are not a rocket scientist, and study the code or collected READMEs for a while, you will probably not manage to do it. That has to change. A good start would be providing an instrument manager, where you can graphically assign which midi channel should get which instruments, without bothering about the details. 4.2 Interoperability There are at least three sequencer-like programs which actively want to talk to aRts: * Brahms - which has been in the CVS for some time now - Jan is working on a new version right now - some aRts->midi support is in right now * Anthem - nothing done right now, Pete definitely wants to have audio stuff like hd recording, and probably aRts->midi support should be added, too * ArtsTracker - an experimental small tracker in kmusic Especially the areas * connecting to aRts * finding (virtual) midi channels and synthesis instruments * assigning instruments to channels * sending timestamped events need to be standarized more for midi. Other interesting fields include * harddisc recording * effect processing Most of this only needs a small amount of code in aRts and the applications, the problem is finding a sophisticated standard. =============================================================================== 5. Not in 2.1 =============================================================================== 5.1 Media interfaces aRts supports streams between objects. Besides the typed (float) audio streams, it also supports untyped byte streams. The idea of extending these towards supporting * which encoding does it have? * what is the size of the frames? * which streams (audio/video) belong together? and others would make a good step towards further modularizing audio/video codecs. It would also allow things like connecting from a decoder to a renderer without knowing which conversions need to be done to make them the same format (a connection manager could find a suitable chain of filters to plug in there). Also http streaming and others could be supported. Not in 2.1 however. -- -* Stefan Westerfeld, stefan@space.twc.de (PGP!), Hamburg/Germany KDE Developer, project infos at http://space.twc.de/~stefan/kde *- _______________________________________________ Kde-multimedia mailing list Kde-multimedia@master.kde.org http://master.kde.org/mailman/listinfo/kde-multimedia