Full text index and search the audio and video audio tracks? Want to search BUILD, Mix, Channel 9 videos in a whole new way? MAVIS is your answer...

Next at Microsoft - Microsoft’s MAVIS audio and video indexing - now available on demand

"I’m a fan of lots of technology from Microsoft Research and MAVIS is a particular favorite – I wrote back it back in Feb 2011 on this blog. MAVIS is an acronym for Microsoft Research Audio Video Indexing System. It enables you to search across audio and video files (think movies, TV, radio) and find the precise moment when a word was uttered. Imagine being able to talk to your TV and say “take me to the point in the movie where Nicholson says you can’t handle the truth” – or asking your TV to find all shows that contained the word “architecture”.

That’s precisely what MAVIS enables by indexing audio streams. If you’re interested in the technology behind this, I’d highly recommend checking out the project page from Microsoft Research or watching a great talk from MIX2010 titled Unlocking Audio/Video Content with Speech Recognition. MAVIS is designed to enable searching of 100s or even 10,000s of hours of conversational speech with different speakers on different topics and has been used by a number of organizations including the State of Washington Digital Archives and ScienceCinema, a website that highlights leading-edge research from the U.S. Department of Energy as well as the European Organization for Nuclear Research (CERN). You can also search all the talks from our recent /BUILD conference.

Now it’s set to be used much more widely. GreenButton is a cloud services company from New Zealand recently announced the availability of GreenButton inCus - a cloud based audio indexing solution powered MAVIS. Running on top of Windows Azure, the service provides audio indexing on demand citing content such as recordings of meetings, conference calls, voice mails, presentations, online lectures, and Internet video as being ideal for the service.

..."

Microsoft Research - MAVIS

"The Microsoft Research Audio Video Indexing System (MAVIS) is a set of software components that use speech recognition technology to enable searching of digitized spoken content, whether they are from meetings, conference calls, voice mails, presentations, online lectures, or even Internet video. A side benefit of MAVIS is the ability to generate automatic closed captions and keywords which can increase accessibility and discoverability of audio and video files with speech content.

MAVIS is now available as a commercial service through a subscription to Greenbutton inCus.

MAVIS Features

Search audio for spoken words - MAVIS uses speech to enable efficient searching for spoken words in audio and video files. The user experience is much like searching for text in documents and on the web. Users type in search terms, the result is a set of links which, when clicked on, will start playing the audio from where those terms were spoken.

Highly accurate audio search Results - Speech recognition is prone to errors which can affect the accuracy of audio search results. The MAVIS technology reduces errors in speech recognition by automatically expanding its vocabulary, and storing word alternatives using a technique referred to as Probabilistic Word-Lattice Indexing, explained in the technical background. These techniques help increase the speech recognition accuracy.

Closed Captions and keyword generation - Closed captions can make audio and video content accessible to the hearing impaired, or translated so that the content can be used by a broader audience in different languages. MAVIS can generate closed captions in the SAMI format. The accuracy of closed captions generated by MAVIS will depend mainly on the clarity of the speaker and background noise. There are a number of subtitle editing tools on the web which can be used to edit the closed captions generated by MAVIS for improved accuracy. Additionally, MAVIS can generate keywords which can better expose media content to search engines such as Bing & Google, and can also be used to categorize your content, or assist in delivering contextual based ads.

..."

Best of all, see it in action, Microsoft Video Web

Being a Channel 9 guy I focused on the Channel 9 search...

And what does anyone and everyone search for the first time they encounter a new search engine? Themselves of course! (Right? I mean... right? Don't they? I hope it's not just me... ;)

Now the coolest part? If you click on the search hit, the video will jump to that time index...

There's also indications as to where in the video the search terms were found, how long they played, etc

Pretty awesome isn't it?

NOTE: If a video doesn't seem to work, give it a couple days... There's DNS/Domain issue that Microsoft has been dealing with this week that could be the issue.