AI software can identify objects in photos and videos at near-human levels

A new AI software program developed by researchers at Google and Stanford University can recognise objects in photos and videos at near-human levels of understanding.

ai software program google stanford university object recognition technology images videos human level understanding

It was only recently that computer systems became smart enough to identify unknown objects in photographs. Even then, it has generally been limited to individual objects. Now, two separate teams of researchers at Google and Stanford University have created software able to describe entire scenes. This could lead to much better and more intelligent algorithms in the future.

Stanford's work, entitled "Deep Visual-Semantic Alignments for Generating Image Descriptions", explains how specific details found in photographs and videos can be translated into written text. Google's version of the technology, in a study titled "Show and Tell: A Neural Image Caption Generator", produced similar results.

While each team used a slightly different approach, they both combined deep convolutional neural networks with recurrent neural networks that excel at text analysis and natural language processing. The programs were able to "learn" from each new interaction, with algorithms enabling the system to improve its accuracy by scanning scene after scene, looking for patterns, and then using the accumulation of previously described scenes to extrapolate what is being depicted in the next unknown image.

"The system can analyse an unknown image and explain it in words and phrases that make sense," says Fei-Fei Li, a professor of computer science and director of the Stanford Artificial Intelligence Lab. "This is an important milestone. It's the first time we've had a computer vision system that could tell a basic story about an unknown image by identifying discrete objects and also putting them into some context."

These latest algorithms are being trained on a visual dictionary – the ImageNet project – with a database of more than 14 million objects. Each object is described by a mathematical term, or vector, that enables the machine to recognise the shape the next time it is encountered. Those mathematical definitions are linked to the words humans would use to describe the objects.

“I was amazed that even with the small amount of training data that we were able to do so well,” said Oriol Vinyals, a Google computer scientist who worked with members of the Google Brain project. “The field is just starting, and we will see a lot of increases.”

In the near term, computer vision systems that can discern the story in a picture will enable people to search photo or video archives and find highly specific images. Eventually, these advances will lead to robotic systems able to navigate unknown situations. Driverless cars would also be made safer. However, it also raises the prospect of even greater levels of government surveillance.

"A group of young people playing a game of Frisbee."

"A person riding a motorcycle on a dirt road."

"A pizza sitting on top of a pan on top of a stove."

Comments

How ad-free subscriptions could solve Facebook

At the core of Facebook’s “well-being” problem is that its business is directly coupled with total time spent on its apps. The more hours you pass on the social network, the more ads you see and click, the more money it earns. That puts its plan to make using Facebook healthier at odds with its finances, restricting how far it’s willing to go to protect us from the harms of over use. The advertising-supported model comes with some big benefits, though. Facebook CEO Mark Zuckerberg has repeatedly said that “We will always keep Facebook a free service for everyone.” Ads lets Facebook remain free for those who don’t want to pay, and more importantly, for those around the world who couldn’t afford to. Ads pay for Facebook to keep the lights on, research and develop new technologies, and profit handsomely in a way that attracts top talent and further investment. More affluent users with more buying power in markets like the US, UK, and Canada command higher ad prices, effectively...

The CC Aurora is actually pretty fun, as far as projectors go

I don’t review projectors. Projectors are boring. Even the good ones. They remind me of vacation slideshows and the film strips we had to watch in health class — neither of which I’m in a particular hurry to revisit in my adult life. That said, I’ve always harbored some germ of a notion that some day I might buy one, to compensate for being one of those weirdos without a TV set. There’s something undeniably appealing about a big screen TV you can break out during movie night and then stash back into the closest of your one-bedroom New York City apartment. XGIMI’s CC Aurora is the closest I’ve seen to fitting the bill — or, for that matter, being a projector that I could actually muster any reasonable amount of excitement about. From the looks of it, it’s kind of the perfect package for the apartment dweller: it’s compact, self-contained with a built-in speaker system and plays nicely with mobile devices. Clearly I’m not alone here, either. The product scored $170,000 on Indi...

Hundreds Of Apps Banned From App Store For Accessing Users’ Personal Information

Hundreds of iOS applications have been pulled out of the App Store, following a report from analytics service SourceDNA , which uncovered a group of applications that were extracting users’ personally identifiable information, including email addresses associated with their Apple IDs, devices and peripheral serial numbers, as well as a list of apps installed on their phone. The applications in question had been using an SDK from a Chinese advertising company called Youmi which was accessing this information by way of private APIs, the report found. Nearly all of the developers were located in China so, for now, this appears to be an isolated incident. However, the larger concern here has to do with how long this activity had been taking place – and what that means in terms of Apple’s App Store review process, given that it hadn’t caught this suspect activity until being alerted to it by a third party. According to SourceDNA, Yo...

Cityscoot raises $50 million for its European electric scooter service

French startup Cityscoot announced a $50 million round of financing yesterday (€40 million). RATP Capital Innovation and Inventure Partners are leading the round, with existing investors Caisse des Dépôts and LeasePlan also participating. Cityscoot operates an electric scooter-sharing service in Paris and plans to expand to other cities in France and Europe. You can now find around 1,600 scooters in Paris or some cities next to Paris. With your phone, you book a scooter in a few seconds and type a code on the scooter to unlock it. When you’re done, you just have to park the scooter and end the ride. And if you live in Paris, chances are you’ve seen countless Cityscoots over the past year or so. They’re completely silent and they come with a helmet in the storage space under the seat. While Cityscoot is leasing its scooters, it’s a capital-intensive industry. For instance, Cityscoot’s (electric) cars change the batteries when they’re about to die so that they don’t have to ...

Uber, Google and other tech employees form Coalition of Black Excellence

When black employee resource groups from a variety of tech companies come together, black magic happens. More specifically, black excellence happens. The Coalition of Black Excellence Week, spearheaded by Uber Litigation Counsel Angela Johnson in collaboration with black ERGs from over 40 tech companies like Facebook, Google, eBay, Lyft and Microsoft, kicks off this Monday in the San Francisco Yay (Bay) Area. The idea for CBE Week came in part from Johnson’s experiences living in Washington D.C., and being able to attend events put on by the Congressional Black Caucus, she told me at Uber’s headquarters this week. “When I moved out to the Bay Area, I really wished there were similar types of experiences for tech,” Johnson said. “And I thought if we could bring together different black ERGs, or diversity and inclusion committees, or people who were interested in some of the issues the black community is passionate about, a lot of positive change and impact could come from that....

Exciting Technology

Search This Blog