CM Notation Typesetter as a multimedia learning tool - A pro

arunk · Post by **arunk** » 11 Apr 2008, 08:35

Dear Friends,

I have a prototype usage of my notation typesetter program which I would like your input on. I am very excited about it - and at the least I can promise that many of you may find it very interesting.

First, I hope Shri. Nandagopal and Ahiri wont mind me taking some liberties here. I have typeset the paTdIp varNam using my software but on top of that this prototype integrates (a snippet of) Ahiri's audio version into the notation. As you play the audio, you can see which swaras are being sung (only the pallavi part is there in the audio). You can even slow down the audio (without losing pitch). So the typeset notation becomes a multimedia presentation using which one can get tremendous insight into the song.

BTW, dont get your hopes too high

It is not like it is decoding the music automatically!! Think of this is an elaborate captioning scheme. All the information (i.e. time indices into the audio file) has to be manually entered.

This was one of my "dreams" of using the typesetter and so needless to say I am excited about the outcome. While I expect the usage to be limited to demonstration and learning purposes, it nevertheless "seems cool".

Here is the link to paTdIp varnam notation with Ahiri's beatiful voice integrated into it: http://arunk.freepgs.com/cmnt_audio_proto/patdip.html

Please note the following:
1. It is a prototype. It has basic functionality. You play the audio and you see the swaras highlighted as the song progresses. You can also click on the swaras (again just the pallavi part) and the audio would seek to it.
2. The initial load will take a bit of time (atleast a few seconds - longer on slower connections)
2. At this point, it will work ONLY on PCs and probably ONLY under IE and Firefox (for Firefox you need the Media Player plugin: http://port25.technet.com/pages/windows ... nload.aspx)
3. As you may see only the pallavi part is integrated into the audio (in fact the audio is a snippet from the full version)

Please let me know what you think.

Arun

mohan · Post by **mohan** » 11 Apr 2008, 09:40

Wonderful!

vasanthakokilam · Post by **vasanthakokilam** » 11 Apr 2008, 12:24

Arun, fantastic. It is a lot of fun of course to follow the swaras along with music. It is quite educational. Congratulations. Very cool indeed. I just love the "seek to the note" feature. I am captivated by that little segment ' d p g ...' in the second line of the pallavi and I kept hitting it to play it again and again.

mnsriram · Post by **mnsriram** » 11 Apr 2008, 14:16

Superb Arun and very very useful. Thank you

knandago2001 · Post by **knandago2001** » 11 Apr 2008, 17:55

Wonderful!

cmlover · Post by **cmlover** » 11 Apr 2008, 18:51

Wonderful dear Wizard!
You have opened a new chapter on 'see and learn CM' (sight and sound).
My hearty congratulations. Do explain how it is done so that other do-it-yourself CM enthusiasts can format Varnams (even geethams) based on notations to assist in effective learning!

I am simply thrilled! (For once I am lost for words

arunk · Post by **arunk** » 11 Apr 2008, 20:00

Thanks all.

cmlover - It is basically an enhancement to the notation typesetter input (of course not part of the official released version (yet) ) - the prototype uses a modified "alpha version" if you will).

You basically have to correlate the time indices to the swaras and indicate it in the typesetter input. So something like:

Code: Select all

Audio: http://mysite/myaudio.mp3
S: s   r   g   m    p   d    n    s'
T: 1 1.3  1.8 2.2  2.8 3.1  3.35 4.02

The "S" indicates swara (of course it can be followed by a L line for lyrics). The new thing here is the "T" line for audio time indices. Here the T line indicates that sa is 1 second into the audio file, ri is at 1.3 etc. Obviously constructing this information is sort of a painstaking process (and hence I expect this is to be use only for demo/learning purposes).

I opened the file in an audio editor (e.g Audacity) - zoomed it in and seeked to various portions to get time indices (at a fairly precise level). This is of course very easy for the portion where Ahiri sings the swaras

. But when it goes to sahityam, one has to listen more carefully. But this is still easy for varnams (and gitams), and for such songs providing indices to every swara is easily possible and practical.

But for krithis - this can be quite challenging to know every swara (unless of course you know the song by-heart and the audio confirms to what you know). And the swaras can be very tightly packed together - although that is where the slower playback rate can help. But still for krithis in general things get a bit dicey in places where there are many anuswaras (so e.g. notating following RRI's style means every turn is codified as a swara and that can be challenging here)

For varnams of course the additional advantage is the notations are fairly well-set (not that much variation in practice). Other usage is lec-dems where little snippets of ragas can be better demonstrated using this.

Arun

cmlover · Post by **cmlover** » 11 Apr 2008, 20:38

Arun
I know when one looks at it , it all looks simple and elegant but the labour that goes into it is mammoth. I trust once it is ready the Music Colleges will take over and train students to input Varnams and Kritis too (properly sung) whence it will become a superb CM learning machine. You can also incorporate tala clicks at the precise moments so that Laya also will be taken care of! More Power to Arun's software!
(Did you look at:
http://rasikas.org/forums/viewtopic.php?pid=77510#p77510

Suji Ram · Post by **Suji Ram** » 11 Apr 2008, 21:55

This is really great Arun!
As you said for varnams this would be wonderful. Also with tala visuals along with short demos this tool can be effective esp for beginners and for those trying to understand the complexity...(ah! the more I learn the more complex it gets).

rajeshnat · Post by **rajeshnat** » 11 Apr 2008, 22:23

Arun
Your software is just empowering whole set of people who aspire to sing, but just dont know how to take the first step. Just brilliant , keep it up.

rajeeram · Post by **rajeeram** » 11 Apr 2008, 23:35

Arun
This is way too cool. The thought and effort that have gone into this project are very very commendable indeed.

This will be a great learning tool for kids. I get my 5-year old son to practice by enticing him to record it on my laptop and see his music 'paint the screen' in Audacity. So if I can get him to learn a new geetham while 'jumping' through the swarams that is notated like this, it would make my job real easy:)

Yes as you say marking the time indices for kritis with a lot of gamakas or fast/superfast brigas will be tricky. And I imagine modelling some artistes' renditions would be exponentially more difficult than others::)

Lots of food for thought

Thanks

arunk · Post by **arunk** » 12 Apr 2008, 00:03

suji, rajesh, rajeeram - thanks!

suji/cml: The tala indicator also in the mix! Yes - good idea! (althoughI first need to fix that prototype so that it uses other images - not sure what kind of copyrights the one I used carry)

rajesh: Now, I expect a "mini katcheri" from you when I come in December

? Maybe all of us novices will take over a jam session

! Although that could spawn an underground group that wishes to save CM, and thus hires a nemesis programmer who engineers a deadly software virus that specifically targets my website rendering this app useless!

rajeeram: Yes. I do think kids would like it. Yesterday, I previewed this to my 7 year old daughter (my "first audience") - and she immediately quipped: so if I now play nApAli SrI rAma (a song she is learning as part of a show) it would show what is being sun and help me

? Of course she thought the software was figuring out stuff automatically!

Arun

vidya · Post by **vidya** » 16 Apr 2008, 00:04

Arun,
One obvious application of this would be to project Pancaratnams or any of the group rendition type stuff on big screen at Tyagaraja Aradhanas etc
Would help people who learnt from multiple sources reconcile it easily plus having not to peek down surreptitiously(or not) at notebooks. Of course some enterprising musician in search of the next innovation would use it to project it at concerts with the added risk of the audience joining in and humming along (at multiple srutis ofcourse) to add to the tala menace.

vainika · Post by **vainika** » 16 Apr 2008, 00:12

I wish I could join the chorus of voices exclaiming in wonder and delight over Arun's latest offering, but have to play the part of disgruntled Mac user denied this privilege.

arunk · Post by **arunk** » 16 Apr 2008, 03:49

. In time, in time (hopefully)! I have a mac too. I need to see what quicktime can do for me (it didnt do enough on the PC)

Arun

arunk · Post by **arunk** » 16 Apr 2008, 03:59

vidya wrote:Arun,
One obvious application of this would be to project Pancaratnams or any of the group rendition type stuff on big screen at Tyagaraja Aradhanas etc
Would help people who learnt from multiple sources reconcile it easily plus having not to peek down surreptitiously(or not) at notebooks. Of course some enterprising musician in search of the next innovation would use it to project it at concerts with the added risk of the audience joining in and humming along (at multiple srutis ofcourse) to add to the tala menace.

Yes indeed. Initially I thought although I would think that a display that gets to a swara-by-swaera level is overkill in that situation and even can be distracting, for pancaratnas one need to know the swaras and sing them and so it would be useful indeed.

I was wondering if it would be a good idea to host such notations on some website (mine to start) for learning (say varnams, gItams, and even krithis)? We can sort of have build an online database. Of couse the audio must be clean and good (fidelity as well as musicality) if we want people to get educated (or atleast use as a reference). It can be people's own effort of otherwise - but audio should have none of the copyright headaches.

Arun

arunk · Post by **arunk** » 05 May 2008, 21:11

Please see the latest entry on my blog (on rItigauLa): http://sunson.wordpress.com/2008/05/05/ ... ritigaula/

I have used this prototype to make some illustrations there (and you dont need to be on a PC for this

)

Arun

vainika · Post by **vainika** » 05 May 2008, 23:33

nice!

knandago2001 · Post by **knandago2001** » 06 May 2008, 10:21

Arun: Very enjoyable, indeed! Ragaratna maalikache illustrates a d n S* n d (raa.... ga.) prayoga, as well as its variation: d n S*r*s* n d (both of which contrast with the "usual" m n n s* or p n n s* ascent). Taamboolava koLLo by Purandara dasa highlights the p ,, s*n n d m (and its variant p , r*s*n n d m) prayoga beautifully. I haven't encountered these phrases in Anandabhairavi - please do let me know if you find them in any composition. Just remembered that Syama Sastri's "ninnu vina marigalada" is sung in ritigaula by MSS and the DKP/DKJ schools.

arunk · Post by **arunk** » 06 May 2008, 17:07

Thanks vainika and nandagopal.

Thanks for the info on rAgaratnamAlikacE - I will pay close attention to it again. Yes - I also forget about ninnuvina marigalada being rendered in rItigowLa by some!

Arun

vasanthakokilam · Post by **vasanthakokilam** » 06 May 2008, 22:48

Fantastic blog post Arun. Using your tool to show the swaras as the song progresses finds its practical use here in such raga illustrations.

Thanks.

arunk · Post by **arunk** » 06 May 2008, 22:57

thanks vk