When it comes to recommending music, there’s one thing that Pandora and Spotify can’t do: read your mind. But Gert Lanckriet, assistant professor at UCSD’s Department of Electrical and Computer Engineering, is developing an application that does exactly that.

“We want this device here to be zero click,” Lanckriet said about the application he’s been working on since 2009. “It plays the right music we want to hear right now without even asking me any type of questions.”

Like the perfect best friend you never had, the app will be able to tell you’re still hurting from that breakup — and find the best tunes to soundtrack your wallowing.

But the application’s mind-reading capabilities won’t be so mysterious; it will utilize smartphone components like the camera, the microphone and the GPS system to analyze characteristics like voice inflection, typing speed and location to tag the user’s mood and surroundings and match those tags with music.

Lanckriet has already developed the algorithm for one step of the complex process: labeling music without having to actually listen to the songs.

“We have algorithms to automatically tag music,” Lanckriet said, describing a  program he already designed back in 2005, that can automatically identify what characteristics a particular song has. “Music is a one-dimensional wave form. You feed that in the computer and
it says rock, dance, guitar. If someone wants to listen to romantic jazz with saxophone and male vocals, we can actually find music and we
can find those songs with those tags.”

He’s tagged over 10,000 songs, and with access to more music catalogues, he could potentially tag the millions of songs available online in places like Spotify and Myspace. He’s even developed a Facebook game to help him speed up the process: “Herdit,” a game in which users describe the music they’re listening to. Over time, “Herdit” will compile the most popular adjectives used to describe songs so Lanckriet can add the tags to his database of music.

For the next component of his project — being able to identify what a person is feeling — Lanckriet found an unlikely (but convenient) collaborator: UCSD’s own medical school.

Even though the med school was working on its own portable mood detection software, Lanckriet did not actually contact the medical school itself.

“It was a coincidence that it happened,” Lanckriet said. “I was introduced to them through somebody. They actually had a grant proposal from NIH to use accelerometers and GPS signals to try to predict the energy expenditures for medical patients.”
Lanckriet’s team and the med school have two totally different goals in mind, but both parties want to find a way to detect someone’s mood and activity easily through a phone application, and each of the groups provides skills the other lacks.

Through the med school, Lanckriet can find out what detailed characteristics of a person’s behavior can be associated with which moods, and he can develop algorithms to place a tag on things like voice inflections and typing speed, which would then be matched up with musical tags.

It works well for both of them; certain components of the software that Lanckriet’s working on could be utilized by doctors whose patients need to be closely monitored.

The music recommendation project has attracted big names like Google, Yahoo, Qualcomm and IBM, all of whom intend to work with Lanckriet and fund his project. He’s also received funding from the National Science Foundation, the National Institutes of Health, the Alfred P. Sloan Foundation and the Hellman Family Foundation. Lanckriet predicts that a basic app will be well under way by June 2012.

Not all of the attention has been positive. Some are concerned with the privacy issues that would arise out of the application, with many
disconcerted by the possibility of third parties (like advertisers) knowing someone’s mood and location and exploiting that information.

“On one side I completely understand that people have a certain reservation about this in terms of privacy,” Lanckriet said. “It’s certainly not our intent to push anybody somewhere where they’re uncomfortable with it. We’re initially just going to work with that opt-in crowd.”

Lanckriet insists that his main concern is finishing the development of the software, but he also predicts that people will be less worried about the privacy issues once they see what the app can do for them.

“If you look at Facebook there’s constantly a discussion about how they invade our privacy, but over time a lot of people do subscribe to it because they see the value of it,” Lanckriet said.

He’s also enlisted the help of undergraduates to start developing simple application interfaces to host the algorithms.

“To show you that we’re very serious about this right now I’m working with a group of undergrads where basically every student is designing an app that will do this,” Lanckriet said.

Lanckriet himself doesn’t identify as an “audiophile” (“I can barely remember names of songs,” he admitted) but it was the social aspect of music that inspired him to take on creating this application.

“At a student barbecue six years ago I met two students over a keg of beer,” Lanckriet said. “They needed a drummer for their band. I played drums, another guy played banjo and we were making music out of the blue. I thought, if you upload this to Myspace nobody’s gonna listen to it.”

That’s when he thought of giving people a simple way of finding music other than what usually plays on the radio, without asking listeners to make any sort of extra specifications.

While there is some merit to the privacy issues presented by his opponents (it is pretty creepy to think that your iPhone might be able to figure out how you really feel about your ex’s new girlfriend), Lanckriet is sure the streamlined and intuitive nature of the program may override the concerns. Not to mention, when you ask for music recommendations, it’ll be nice to know that at least someone knows you don’t appreciate dubstep.