Automated Raga Identifier

[bhavarasa]

I recently came across an iPhone app called SoundHound. It "listens" to a clip of a song that's being played and searches its database to come up with the name of the song. It works mostly on Western music (multiple genres) but it was able to recognize a couple of CM krithis as well, as they were from internationally released albums of popular artists.

The question I have is whether or not it is possible to "program" a computer to "listen" and identify the raga of a CM song. I saw an old thread here titled "How we detect ragas". Interesting discussion. I wonder if we can use our collective intelligence to create artificial intelligence.

If I were to attempt to break it down into logical steps that a computer can follow, it would be something like this:
1. Identify anchor notes (sa/Pa/SA or sa/ma/SA et al.)
2. Identify individual notes
3. Identify note sequences (aro/avaro)
4. Search database for matching rAgA and report output

Now, I realize it sounds very simplistic but I just want this to a starting point for the discussion.

Thanks in advance!!

[ShrutiLaya]

I'm no expert, but I think the problem has great similarities to speech to text conversion (if not speech recognition). There too, you have the equivalent of different sruthis (i.e you can't just associate a particular frequency with a particular letter of the alphabet, it depends on the speaker), different accents, different combinations. It is a combination of brute force memory (know lots of words / krithis) and technique (statistical or other mathematical manipulations / raga flashing in the subconscious). And yet, many software packages can do a fairly good job of speech->text right out of the box. Even my car recognizes spoken commands, and I tried putting on fake accents and changing speaking speeds without being able to confuse it! (It does a lot worse when I try to spell the alphabet to input street names into the GPS - confuses p/b/t etc.,).

We need someone who is knowledgeable in both Carnatic music and Speech recognition/conversion to take a look at this problem. Apparently basic speech recognition software is available as open source ..

- Sreenadh

[VK RAMAN]

speech pathology schools - normally under school of medicine - do work on pitch and recognizing ragam may not be that far away

[bhavarasa]

The iPhone app that I mentioned, SoundHound, recognizes the song based upon the tune and not the words. I verified this by playing Strauss' Frühlingstimmen and it was able to pick up on it correctly.

But while SoundHound detects the song, what we are discussing is detecting the raga. So do we have to necessarily detect spoken (sung in this case) words OR just the frequencies in order to detect the raga?

[vasanthakokilam]

Shazam is another such program which has been around for the past 8 years. Shazam was a pioneer in this technology. Shazam actually recognizes indian movie songs as well.

They have a huge library of songs ( 8-10 million or so ). For each song, they have their own proprietary method of converting the song to a unique numerical signature;fingerprint. That fingerprint is unique to that song. For recognition, it converts a little bit of the song to the fingerprint using the same method and searches its database for a pattern match and finds the song.

Key challenge is, how do you convert a song which is quite rich in musical information to a simple signature. Shazam's basic approach is to ignore pretty much everything in the song except the high energy intense portions of a song relative to the other notes around that section. ( based on a graph of amplitude, frequency and time ). Its matching success rate is quite high. It can matches different versions of the same song, say covers of the same song by different artists.

I guess such a 'coarse' approach to raga identification may work. It may not actually find the raga directly but if you play a song that it knows about, it can first match the song and then read off the raga from its database. That should be trivial since they have solved the tough problem of matching the song.

If it has songs of a particular raga in its database, it will be interesting to make it match that same raga for a song that is not in the database.

There are programs that can plot a spectrogram of frequencies. We played around with a program like that in this thread: http://www.rasikas.org/forums/viewtopic.php?f=9&t=4339

So at least theoretically, there is a way to identify the various frequencies, make some guesses on which frequency is Sa (or specify it to the program ) and try to match it to a raga.

[arvindt]

The first problem is to identify the tonic shadjam which could be done by Fourier analysis
if there is a loud enough tambura shruti in the clip.

After that, I think even just looking at the dwell times on various frequencies (i.e. which notes
are rested on more as nyasa and which aren't) would give decent first order predictions for
common ragas. More sophisticated techniques like recognizing "signatures" for various
gamakas would be a refinement to this techique.

This project would be cool because it would help characterize a raga more formally and
accurately than can be done in words. Still would be a very long way to go to the "generation"
problem i.e. that of generating an alapana for a raga because that requires encoding
very complicated rules for scale, gamaka, bhava, voice modulation/texture and dwell times.

[hariniraghavan]

I am sure it can be done with the help of instruments. Once we know the gamakas and other nuances of a particular raga very well, it should be possible to reproduce the same with the help of i nstruments particularly the veena. If somebody is coming forward with this idea I would be interested in helping to best of my capacity.
Harini.

[Shyamala Rangarajan]

It will be of interest to know that the well known violin artists Shri Ganesh and Shri Kumaresh are collaborating with the Research Scholars and Students in SASTRA University, Thanjavur, in developing Digital models, applicable to Carnatic Music. Can 'gamaks' of Carnatic Music be 'digitally defined' on different parameters so that more objectivity is introduced in Carnatic Music? Research is being undertaken to find answers for this question. Please refer to their web site gamaka.org.