Deep understanding of artificial intelligence & Machine Learning at Apple (part 1)

Tram Ho

A close look at Apple’s use of artificial intelligence and Machine Learning .

On July 30, 2014, Siri successfully performed a brain transplant.

Three years earlier, Apple was the first major technology company to integrate smart assistants into its operating system. Siri was a variant of a standalone application that Apple acquired in 2010. The initial reviews surprised many people, but in a few years, months later, users could gradually lose patience because of the common Siri. wrong translation of command lines.

Therefore, at the end of July 2014, Apple turned the feature of identifying Siri’s voice into neural network for US users (global launch on August 15, 2014). Some techniques are still in operation but now, Siri systems mainly take advantage of Machine Learning technologies, including Deep Neural Networks (DNN), convolutional neural networks, and long-term networks. short-term memory units (LSTM), gated recurrent units (GRU) and n-grams. When users update, Siri’s interface remains constant but it will be supercharged by deep learning.

These advances may affect competitive strategy, so Apple does not publish the entire development process widely. If users notice that Siri is making fewer and fewer errors and with Apple, this is a very amazing result.

Eddy Cue – Senior Vice President of Internet Software and Services

The story of Siri’s transformation revealed for the first time in this article may cause the tech world to study artificial intelligence to ponder. Apple has quietly “trained” this technology, and until recently, when Apple recruited AI characters and made some high-end insights, experts said that Apple had proceeded. Slow steps in the hot battle possess the most powerful AI tool.

“At Google, Facebook, Microsoft, you own outstanding people about Machine Learning ” – Oren Etzioni of Allen Academy on AI said – “Yes, Apple actually hired some experts but 5 leaders about Machine Learning are Who is working for Apple? Apple has voice recognition but we don’t know whether ( machine learning ) still supports Apple somewhat. Please tell me which parts of your product are using the best Machine Learning ”

Earlier this month (August 2016), I spent a day in One Infinite Loop’s meeting room at Cupertino headquarters to learn Apple’s AI and Machine Learning activities with top experts (Eddy Cue, Phil Schiller – Senior Vice President of Global Marketing and Craig Federighi – Vice President of Software Engineering) and two other Siri scientists. They sent me a 2-page document listing Apple products and services using Machine Learning.

The message here is: We have already joined the game. We will do it our own way.

If you are an iPhone user, you certainly have access to Apple’s AI, from Siri’s acumen when identifying what you ask “she”, so that the iPhone can identify the caller whether that person in your contacts (but emailed you recently). It’s also when you scan the screen to get a streamlined list of apps that will most likely open next or remind you of the meeting time you’ve never dropped into your calendar. That’s when the location map shows up, leads you to the pre-booked hotel, before you type in the location or the iPhone will indicate the parking place even if you don’t ask it. These are all techniques that have been or will be applied from deep learning and neural nets.

Yes, there is something called “Apple brain” in the iPhone.

According to Apple experts, Machine Learning has been applied to all products and services of “apple houses”. Apple uses deep learning to detect phishing scams on Apple store, expanding the battery life between charges on all devices, discovering the most useful suggestions from thousands of reports of beta testers. Machine Learning also helps Apple choose stories for you, determine whether Apple Watch users are practicing sports or just walking. In particular, Machine learning also recognizes the face and location in the photo, determines whether to turn off the beloved Wi-Fi signal to switch to the mobile network, to help you know how good a movie is. Since then Apple can quickly compile your snapshots and videos into a mini movie with just one touch of a button. Apple rivals also do the same but according to “apple” experts, there is no AI provider that has just met the above requirements and kept the security with Apple. Of course, no one creates products like Apple.

AI is not something new for Apple: at the beginning of the 1990s, Apple used some Machine Learning techniques in handwriting recognition products ( Newton ?). The remainder is the feature to convert Chinese characters into text or identify input letter by word from Apple Watch users when they scribble a certain message on the phone screen (both features This is produced by the same team of Machine Learning engineers. Of course, early on, Machine Learning was only at a primitive level and Deep Learning did not make a big sound. Today, AI techniques are becoming a trend and Apple will be very upset if someone thinks their learning ability is poor. In a recent speech, Apple also mentioned that studying the AI & the top leaders is gradually building it.

“In the past 5 years, we’ve been studying AI very well,” Phil Schiller said. “Our devices are getting smarter at faster speeds, especially the A-series chips. Apple designed it. Backends are also smarter, faster than before and everything we do has some reason to connect with each other. This method also requires more and more techniques of Machine Learning because there are many things to learn .. ”

Although Apple is bear-hug (1 way of pricing aims to “buy” for a company, a shareholder or a group of shareholders, to gain control of the company or to take advantage of its own initiative. ) Machine Learning technology but experts think that this is a quite normal case in business. Cupertino’s grandfather watched Deep Learning and Machine Learning is the latest technology in the steady flow of breakthrough technologies. Yes, it has the ability to convert but there is nothing more than other technologies such as touch screens, flat panels or object-oriented programming languages. In Apple’s view, no matter what other companies say, Machine Learning is not the last line. “Over the years, there have been many other technologies that change the way we interact with devices,” Cue said. And no one in Apple wants to mention the “scary” / scary speculations that always appear in AI discussions. As you can see, Apple hasn’t made it clear whether they’re developing self-driving cars or their own Netflix version. But the team at Apple made it clear that Apple did not study Skynet .

“We use these technologies to do what we always want to do, not what we have to do,” Schiller said. “And new things we can’t do yet.” OK. It will be Apple-style technology because it grows within Apple and affects the way we create products. ”

It can be said that AI has formed the whole experience of using Apple ecosystem. The idea of ​​forming AI is that Apple is constrained to search engines (tools that transfer data to support neural networks) and perseverance but lack flexibility in user information security. . But it turns out Apple has found a way to overcome both challenges.

How big is this “brain”, dynamic cache that can activate Machine Learning on iPhone? When I mentioned this to Apple, they said: about 200 megabytes, depending on how much information is stored. This personal information includes app traffic, interactions with others, handling related to net neural, speech modeler and “natural language event modeling.” There is also data used in neural nets to implement object identification, face recognition and landscape classification features (classification scene).

And according to Apple, everything is done.

Although Apple has never explained it all   The company’s efforts with AI, but I still understand how the company distributes Machine Learning experts throughout their organization. The Product team is encouraged to use Machine Learning to solve problems and invent new features in individual products. “We are not the focus team that holds the role of Machine Learning Temple at Apple” – Craig Federighi shared – “We try to bring Machine Learning to the teams by applying it to the user experience. it’s correct”.

How many people at Apple are working with Machine Learning ? “Very much,” Federughi said. The main thing about Apple is that Machine Learning is being developed by many people who have never been trained in this field before joining Apple. “We recruit talented people in the fields of mathematics, statistics, programming languages, coding. As a result, many talents have successfully converted knowledge in those fields to Machine Learning . Although we have recently hired a lot of Machine Learning personnel, we are still looking for individuals with appropriate talents and talents ”.

From left to right, Craig Federighi – Senior Vice President for Software Engineering is listening to Alex Acero – Siri Senior Director – presenting the voice recognition software at Apple headquarters.

This is a necessary approach because the tendency to keep Apple secret is in a disadvantage compared to competitors – companies that encourage their computer scientists to widely reseach to the world. . “We often study the natural selection process of individuals who are interested in group work to bring a good product, rather than publishers (publishing) as the main motivation,” Federighi said. If in the process of developing Apple products, the scientists accidentally created a breakthrough in the industry is really great. “But we look to the vision of the end result,” said Eddy Cue.

Besides, some individuals come from Apple’s validation process. “We recently bought from 20 to 30 companies in 1 year. These are quite small companies and really need to hire resources, ”said Eddy Cue. When Apple bought an AI company, it was not a story “we own a group of Aid Machine Learning, develop that team”. On the contrary, “we look for people who are talented in AI and have to really focus on delivering great experiences”

The most recent transaction is Turi – a Seattle-based company received by Apple for $ 200 million. Turi built a toolkit Machine Learning , which is comparable to Google’s TensorFlow. As a result, this transaction blamed speculation that Apple would use Turi with the same purpose as both internal and programmers. “They have a lot in common with Apple in terms of technology and personnel views,” Eddy Cue said. In the next 1 or 2 years, we will know what will happen as what Siri has shown to Cue’s potential (not Eddy Cue!) – a startup that Apple has adopted in 2013.

No matter where the talent comes from, Apple’s AI structure still supports this big guy to develop products and features that have never been done before. It also changes Apple’s product schedule. “At Apple, there is no limit to the development of new ideas,” Schiller said. ” Machine Learning allows us to do what we previously denied.” Machine Learning is gradually entering the decision making process of our next products ”

An example is the Apple Pencil that comes with the iPad Pro. To be able to integrate the high-tech stylus version, Apple must solve the fact that when writing on the device, the human hand must always rub on the touch screen, adversely affecting the digital elements. Thanks to the Machine Learning model “palm rejection”, the touch screen can clearly distinguish the difference between scanning, touching and input from a pen. “If this feature does not work well, the iPad Pro will no longer be the right piece of paper to write – and Pencil is not a good product either,” Federighi said. If you love Apple Pencil, thank Machine Learning .

Source: Techtalk via Backchannel (continued)

Share the news now

Source : Backchannel