Connect with us


Using artificial intelligence to improve video organization



Netra Artificial Intelligence

Netra uses artificial intelligence to help companies understand and organize their video content. Credit: Christine Daniloff, Massachusetts Institute of Technology

Netra, co-founded by Shashi Kant SM ’06, uses artificial intelligence to help companies sort and manage video content.

At any given moment, many thousands of new videos are posted on sites such as YouTube, TikTok, and Instagram. An increasing number of these videos are being recorded and broadcast live. But tech and media companies still can’t figure out what’s going on in all of this content.

Now Massachusetts Institute of Technology Netra, founded by a graduate, uses artificial intelligence to improve large-scale video analysis. A company system can identify actions, objects, emotions, locations and more to organize and present video context in new ways.

Companies use Netra’s solution to group similar content into mainstream videos or news segments, celebrate nudity and violence, and improve ad placement. In the advertising space, Netra helps ensure videos are matched with relevant ads so that brands can opt out of tracking individual people, leading to privacy concerns.

“The industry as a whole is turning to content-based advertising, or what they call interest-based advertising, and moving away from cookie and pixel tracking, which has always been a little creepy,” Netra co-founder and CTO Shashi Kant S. ’06 He speaks.

Netra also believes it is improving video content search. After video processing in Netra system, users can start searching by keyword. From there, they can click on the results to see similar content and find more and more specific events.

For example, the Netra system can process videos from a baseball season and help users find all singles. By clicking on certain pieces to see more similar ones, they can also find all the singles that nearly went out of order and made fans scream angrily.

“Today, video is by far the largest information resource,” says Kant. “It is several orders of magnitude superior to the text in terms of information richness and size, but no one even touched it with a search. This is the whitest white space. “

Following the vision

Internet pioneer and MIT professor Sir Tim Berners-Lee has long worked to improve the ability of machines to recognize data on the Internet. Kant researched under the direction of Berners-Lee as a graduate student and was inspired by his vision of improving the way machines store and use information.

“The Holy Grail for me is a new paradigm in information retrieval,” says Kant. “I believe that Internet search is still 1.0. Even Google has 1.0. This was the vision of Sir Tim Berners-Lee’s Semantic Web Initiative, and this is what I learned from that experience. “

Kant was also a member of the winning team in the $ 100,000 MIT Entrepreneurship Competition (MIT $ 50,000 at the time). He helped write the computer code for a solution called the Active Joint Brace, which was an electromechanical orthopedic device for people with disabilities.

After graduating in 2006, Kant founded Cognika, which used AI in its solution. AI continued to have a bad reputation for being overconceived, which is why Kant used terms like cognitive computing when he offered his company to investors and clients.

Kant founded Netra in 2013 to use AI to analyze video. He has to deal with the opposite end of the hype spectrum these days, with so many startups claiming to use AI in their solutions.

Netra is trying to overcome the hype by demonstrating its system. Netra can quickly analyze videos and organize content based on what happens in different clips, including scenes in which people do similar things, express similar emotions, use similar products, and more. Netra’s analysis generates metadata for various scenes, but Kant says the Netra system provides much more than keyword tagging.

“We work with embedding,” Kant explains, referring to how his system categorizes content. “If there is a scene in which someone makes a home run, there is a certain signature in it, and for that we create an attachment. An embedding is a sequence of numbers or “vector” that captures the essence of a piece of content. Tags are just human readable representations of this. So we’ll train a model that detects all the home runs, but under the cover there is a neural network that creates an attachment of this video, and that distinguishes the scene from a walk or a walk. “

By defining relationships between different clips, Netra allows customers to organize and search their content in new ways. Media companies can determine the most exciting moments of sporting events based on the emotions of the fans. They can also group content by topic, location, or whether the clips contain sensitive or disturbing content.

These capabilities are essential for online advertising. An advertising company representing a brand such as Patagonia can use the Netra system to place Patagonia advertisements alongside travel content. Media companies may offer ad space for brands such as Nike for videos featuring sponsored athletes.

These capabilities help advertisers comply with new privacy practices around the world that place restrictions on the collection of data about individuals, especially children. Targeting specific groups of people with ads and tracking them online is also controversial.

Kant believes Netra’s AI engine is a step towards giving consumers more control over their data, an idea long advocated by Berners-Lee.

“This is not a realization of my CSAIL work, but I would say that the conceptual ideas that I pursued in CSAIL are embodied in the Netra solution,” says Kant.

Transforming the way information is stored

Netra currently counts some of the largest media and advertising companies as its clients. Kant believes that Netra may one day help anyone find and organize the growing ocean of video content on the Internet. To this end, he sees that the Netra solution continues to evolve.

“Search hasn’t changed much since it was invented for the Web 1.0,” says Kant. “There is a lot of link searches right now. In my opinion the links are outdated. You don’t want to visit different documents. You want the information from these documents to be combined into something contextual and customizable, including only the information you need. “

Kant believes that such contextualization will significantly improve the way information is organized and exchanged on the Internet.

“It’s about relying less and less on keywords and more and more on examples,” Kant explains. “For example, in this video, if Shashi makes a statement, is it because he is crazy, or is it something else? Imagine a system that could say, “This other scientist said something similar to support this statement, and this scientist answered this question in a similar way.” For me, such things are the future of information retrieval, and this is the passion of my life. That’s why I came to MIT. This is why I have spent a decade and a half of my life fighting artificial intelligence, and I will continue to do so. “

Originally reported by Source link

The featured images are, as they appear on the original report.


New device draws energy from your sweaty fingers while you sleep



New device draws energy from your sweaty fingers while you sleep

This video shows the left hand with 4 BFCs wrapped around four separate fingers to collect energy from multiple fingers at the same time. Credit: Lu Yin.

Feeling very sweaty from the summer heat? Don’t worry – not all of your sweat should be wasted. In an article published on July 13 in the magazine Joule, researchers have developed a new device that collects energy from sweat in all areas of your fingers. This device is by far the most efficient body energy harvester ever invented, producing 300 millijoules (mJ) of energy per square centimeter with no mechanical energy required during 10 hours of sleep and an additional 30 mJ of energy when pressed once. finger. The authors say the device represents a significant step forward in self-contained wearable electronics.

“Usually you want the maximum return on your energy investment. You don’t want to spend a lot of energy on exercise to get just a little energy back, ”says senior author Joseph Wang (@JWangnano), professor of nanoengineering at the University of California, San Diego. “But here we wanted to create a device adapted to everyday activities that requires almost no energy – you can completely forget about the device and go to sleep or do desktop work, such as typing, but still continue to generate energy. You can call it the “power of inaction.”

This image shows a small hydrogel (right) collecting sweat from a fingertip for a Vitamin C sensor (left) and then displaying the result on an electrochromic display. Credit: Lu Yin.

Previous sweat-based energy devices required intense exercise, such as a lot of running or cycling, before the user was sweaty enough to activate energy production. But the large amount of energy consumed during exercise can easily neutralize the energy produced, often resulting in less than 1% energy return on investment.

On the contrary, this device belongs to the category of energy collectors, which the authors call the “holy grail.” Instead of relying on external, irregular sources like sunlight or movement, all it needs is finger contact to harvest over 300 mJ of energy while sleeping – which the authors say is enough to power a small wearable. electronics. Since no movement is required, the relationship between collected and deposited energy is essentially infinite.

It may seem strange to choose your fingertips as a source of sweat, say, under your armpits, but in fact, fingertips have the highest concentration of sweat glands compared to other parts of the body.

This video shows the process of winding a BFC around your fingertip with a stretch waterproof film. Credit: Lu Yin.

“The production of more sweat on our fingers has probably evolved to help us better grip objects,” says first co-author Lu Yin (@YinLu_CLT), a nanoengineering graduate student at Wang’s lab. “The rate of perspiration on a finger can be as high as several microliters per square centimeter per minute. This is significant compared to other areas of the body, where the level of sweating can be two or three orders of magnitude less. “

The device developed by the researchers in this study is a type of energy harvester called a biofuel cell (BFC) and works by lactate, a compound dissolved in sweat. From the outside, it looks like a simple piece of foam, connected to a circuit with electrodes, each of which is attached to the pad of your finger. The foam is made from carbon nanotube material and the device also contains a hydrogel to help absorb sweat as much as possible.

“The size of the device is about 1 centimeter squared. Its material is flexible too, so you don’t have to worry about being too stiff or weird. You can wear it comfortably for a long period of time, ”says Yin.

A series of electrochemical reactions take place inside the device. The cells are equipped with a bioenzyme at the anode, which oxidizes or removes electrons from lactate; a small amount of platinum is deposited on the cathode to catalyze a reduction reaction in which an electron converts oxygen to water. Once this happens, electrons flow from the lactate through the circuit, creating an electric current. This process occurs spontaneously: as long as there is lactate, no additional energy is required to trigger it.

Apart from but in addition to the BFC, piezoelectric generators are connected to the device, which convert mechanical energy into electricity to collect up to 20% of additional energy. Based on the natural squeezing movement of fingers or everyday movements such as typing, these generators helped to generate additional energy from almost any work: one finger press once per hour required only 0.5 mJ of energy, but produced more than 30 mJ of energy, i.e. e. 6000% return on investment.

Researchers have been able to use the device to power efficient vitamin C and sodium detection systems, and they are optimistic about improving the device with even greater potential in the future that could make it suitable for health and wellness applications such as glucose. meters for people with diabetes. “We want to make this device more integrated into wearable forms such as gloves. We are also exploring the possibility of wireless connectivity to mobile devices for enhanced continuous sensing, ”says Yin.

“There is a lot of exciting potential,” says Wang. “We have ten fingers to play with.”

Read more about this study. The wearable turns your fingertip into an energy source

Reference: “Passive Sweating Biofuel Cell: High Return on Investment” Muyang Lin, Mengzhu Cao, Alexander Trifonov, Fangyu Zhang, Zhiyuan Lu, Ja-Min Chung, Sang-Jin Li and Sheng Xu, July 13, 2021, Joule
DOI: 10.1016 / j.joule.2021.06.004

This work was supported by the UCSD Wearable Sensors Center.

Originally reported by Source link

The featured images are, as they appear on the original report.

Continue Reading


Farming robots are the future – we must prepare now to avoid dystopia



Dystopian Farm Robots

This illustration shows the scenario of a utopian farming robot. Credit: Natalis Lorenz.

This is no longer science fiction, farm robots are already here – and they have created two possible extremes for the future of agriculture and its impact on the environment, says agricultural economist Thomas Daum in an article published July 13, 2021 in Science & Society. Journal Trends in ecology and evolution… One of them is a utopia, in which entire parks of small intelligent robots work in harmony with nature to produce a variety of organic crops. Another is a dystopia in which large, tractor-like robots conquer the landscape with heavy machinery and artificial chemicals.

He describes the utopian scenario as a mosaic of rich green fields, streams, wild flora and fauna, where fleets of small robots powered by sustained energy flutter around the fields, their buzzing mingling with the chirping of insects and the song of birds. “It’s like a Garden of Eden,” says Daum (@ThomDaum), a research associate at the University of Hohenheim in Germany who studies agricultural development strategies. “Small robots can help conserve biodiversity and combat climate change in ways that have never been possible before.”

He suggests that a utopian scenario that is too laborious for conventional farming, but possible with robots working around the clock, 7 days a week, is likely to benefit the environment in many ways. Plants would be more varied and the soil richer in nutrients. Thanks to micro-spraying of biopesticides and laser weed removal, nearby water, insect populations and soil bacteria will also become healthier. Organic crop yields, which are now often lower than traditional crop yields, will be higher and the impact of agriculture on the environment will be greatly reduced.

Dystopian farm robots

This illustration shows the scenario of a dystopian farming robot. Credit: Natalis Lorenz.

However, he believes that a parallel future with negative environmental consequences is quite possible. In this scenario, he says, large but technologically crude robots will bulldoze the natural landscape, and multiple monocultural cultures will dominate the landscape. Large fences would isolate people, farms and wildlife from each other. Once people are removed from farms, agrochemicals and pesticides can be used more widely. The ultimate goals will be structure and control: qualities that these simpler robots excel at, but which are likely to have detrimental effects on the environment.

While he notes that it’s unlikely the future will be limited to pure utopia or pure dystopia, by creating these two scenarios, Daum hopes to spark a conversation in what he sees as a crossroads in time. “Both utopia and dystopia are possible from a technological point of view. But without the right policy barriers, we could end up in a dystopia without even wanting to, if we don’t discuss it now, ”says Daum.

But this impact is not just limited to the environment – it affects normal people as well. “Robotic farming can also specifically impact you as a consumer,” he says. “In utopia, we don’t just grow crops – we have a lot of fruits and vegetables, the relative prices of which will fall, so a healthier diet will become more affordable.”

The small robots described in Daum’s utopian scenario would also be more suitable for small farmers who would find it easier to afford or share them through services like Uber. On the contrary, he argues that a family farm is less likely to survive in a dystopian scenario: only large producers, he says, will be able to manage huge tracts of land and high costs for large machinery.

In parts of Europe, Asia and Africa, where there are currently many small farms, deliberate efforts to implement a utopian scenario offer clear advantages. The situation is more difficult in countries such as the United States, Russia or Brazil, which have historically been dominated by large farms producing large volumes of low-value grains and oilseeds. There, small robots that are less efficient at performing energy-intensive tasks like threshing corn may not always be the most cost-effective option.

“While it is true that the preconditions for small robots are more complex in these areas,” he says, “even with large robots – or a mixture of small and large ones – we can take steps towards utopia with practices like interbreeding, having hedges. agroforestry and the shift from large farms to smaller plots of land owned by large farmers. Some of these methods may even pay off to farmers when robots can do their job as previously unprofitable methods become profitable. ”

To do this, Daum said, you need to act now. While some aspects of the utopian scenario, such as laser weeding, are already developed and ready for widespread adoption, funding must go to other aspects of machine learning and artificial intelligence in order to develop robots intelligent enough to adapt to complex unstructured farming systems. Policy changes are also needed and can take the form of subsidies, regulations, or taxes. “In the European Union, for example, farmers receive money when they perform certain landscape services, such as growing many trees or rivers on their farms,” he says.

While it may seem like a dystopian scenario is more likely, it is not the only way forward. “I think utopia is achievable,” says Daum. “It won’t be as easy as a dystopia, but it’s quite possible.”

Link: “Farming robots: ecological utopia or dystopia?” Thomas Daum, July 13, 2021 Trends in ecology and evolution
DOI: 10.1016 / j.tree.2021.06.002

This work was supported by the “Companion Research Program for Agricultural Innovation”, which is funded by the German Federal Ministry for Economic Cooperation and Development (BMZ).

Originally reported by Source link

The featured images are, as they appear on the original report.

Continue Reading


Dynamic control of THz wavefronts due to rotation of layers of cascade metasurfaces



Metadevice for Dynamically Controlling THz Wavefronts

Meta-device for dynamic control of THz wavefronts by rotating layers of cascade metasurfaces. Credit: Shanghai University.

Cascading metasurfaces for dynamic control of THz wave fronts

Electromagnetic (EM) waves in terahertz (THz) mode are used for critical applications in communications, security imaging, and bio and chemical sensing. This widespread applicability has led to significant technological progress. However, due to the weak interaction between natural materials and THz waves, conventional THz devices are usually cumbersome and ineffective. Although ultra-compact active devices in the THz range do exist, modern electronic and photonic approaches to dynamic control are ineffective.

Recently, the rapid development of metasurfaces has opened up new opportunities for creating highly efficient ultra-compact devices in the THz range for dynamic wavefront control. Ultra-thin metamaterials formed by subwavelength planar microstructures (i.e., metaatoms), metasurfaces allow tuning optical responses to control the fronts of electromagnetic waves. By creating metasurfaces that have certain predefined phase profiles for transmitted or reflected waves, scientists have demonstrated exciting wave manipulation effects such as abnormal light deflection, polarization manipulation, photon spin hall and holograms.

Dynamic beam steering metaservice

Demonstration of the dynamic beam steering meta-device: (a) Schematic of the meta-device, which consists of two layers of transmissive metasurfaces aligned with a motorized turntable. (b) top view (left) and (c) bottom view (right) of a SEM image of the fabricated meta-device. (d) Diagram of the experimental setup shown to characterize the meta-device. (e) Experimental and (f) simulated far-field scattering power distribution with a meta device illuminated with 0.7 THz LCP light and evolving along path I at different times. (g) Evolution of the directions of the transmitted waves on the sphere of direction k when the meta device moves along Path I and Path II, with the solid line (asterisks) denoting the results of the simulation (experiment). Here, the blue area denotes the solid angle for beam steering coverage. Credit: X. Cai et al., Doi 10.1117 / 1.AP.3.3.036003.

Moreover, the integration of active elements with individual meta-atoms within passive metasurfaces allows the creation of “active” meta-devices that can dynamically manipulate the fronts of electromagnetic waves. While active elements in deep subwavelengths are easy to find in microwave mode (e.g. PIN diodes and varactors) and successfully contribute to active meta-devices for beam steering, programmable holograms, and dynamic imaging, they are difficult to create at frequencies above THz. … This difficulty stems from size limitations and significant ohmic losses in electronic circuits. Although terahertz frequencies can drive terahertz beams uniformly, they usually cannot dynamically manipulate terahertz wave fronts. Ultimately this is due to the lack of local tuning capabilities at subwavelength depth scales in this frequency domain. Therefore, developing new approaches to avoid local customization is a priority.

As reported in Advanced PhotonicsResearchers from Shanghai University and Fudan University have developed a general structure and meta-devices to achieve dynamic control of THz wave fronts. Instead of locally controlling individual meta-atoms in the THz metasurface (for example, via a PIN diode, varactor, etc.), They change the polarization of the light beam using rotating multilayer cascade metasurfaces. They demonstrate that rotating different layers (each exhibiting a specific phase profile) in a cascade meta-device at different speeds can dynamically change the effective Jones matrix property of the entire device, achieving unusual manipulations with the wavefront and polarization characteristics of terahertz rays. Two meta-devices are demonstrated: the first meta-device can effectively redirect a normally incident THz beam for scanning in a wide range of solid angles, and the second can dynamically manipulate both the wavefront and polarization of the THz beam.

This paper proposes an attractive alternative way to achieve inexpensive dynamic control of THz waves. The researchers hope this work will inspire future applications of terahertz radars as well as bio and chemical sensing and imaging.

Reference: “Dynamic control of terahertz wavefronts with cascading metasurfaces” Xiaodong Tsai, Rong Tang, Haoyang Zhou, Qiushi Li, Shaoji Ma, Dongyi Wang, Tong Liu, Xiaohui Lin, Wei Tang, Qiong He, Shii Xiao, and Lei Zhou, June 26 … 2021, Advanced Photonics
DOI: 10.1117 / 1.AP.3.3.036003

Originally reported by Source link

The featured images are, as they appear on the original report.

Continue Reading