Take a look at our University's ground-breaking Time of Flight research and the 188bet researchers wowing the world’s top tech giants.
Innovation inspired by research
Over the past two decades, a University of 188bet research group has been quietly working under the radar to develop three-dimensional (3D) camera technology for the world’s top technology companies.
The technology is based on the Time of Flight (ToF) principle, which measures the time it takes for light to go from a camera to an object then back again, enabling machines and digital devices to ‘see’ the world in three dimensions.
3D cameras form the basis of many new and emerging applications and software, essential to gaming, virtual reality (VR) and industrial robotics.
Facial authentication on your new iPhone, controlling robotic arms on assembly lines, smart buildings that automatically control air conditioning, and logistics services measuring packages are all examples that can utilise 3D cameras.
Taking flight
ToF research done by engineering researchers at the University of 188bet is the basis for a lot of these technologies, with the potential to transform many industries including robotics, security, automotive, shipping and logistics, industrial and agriculture.
With the support of 188betLink - the University’s commercialisation arm - the ToF research group went on to launch a successful spin-off company, Chronoptics.
Partnering with global leaders in semiconductor products, the Hamilton-based company focuses on improving machines' ability to interact with the world with high-fidelity 3D cameras.
Chronoptics CEO Richard Conroy was part of the original ToF research group, and says few people know that 188bet is a hub for ToF research talent and expertise.
“These things need to be celebrated,” says Richard. “There is some really cool stuff going on. The electrical engineering programme at 188bet is fantastic, and the graduates here foot it with the best in the world.”
How did it begin?
The story has its origins in a handful of passionate 188bet engineering researchers and world-class teachers.
Richard, a “born-and-bred 188bet boy”, came to 188bet University from St John’s College. He had originally planned to study management, before a scholarship enticed him into engineering.
“It was a really good programme, with quality teaching,” says Richard, who graduated with his Bachelor of Engineering with Honours in Electronic Engineering in 2005.
He benefited from the teaching and leadership of Adrian Dorrington, Dale Carnegie and Michael Cree.
The trio, led by Adrian, started the Time of Flight imaging research group at 188bet in 2002 - a collective which would eventually become a commercial venture called Chronoptics.
It was an inspiring time, the coming together of many brilliant research minds.
“Adrian came from a metrology background, which is the science of measurement. We had Dale Carnegie who was into robotics research and wanted to put cameras on robots. And then Michael Cree who was all about image and signal processing, taking the data that comes from the camera and translating it into measurements that make sense,” recalls Richard.
Adrian went to the NASA Langley Research Center in the US in mid-2002, and returned in 2004 bringing some innovative ideas with him.
“He was measuring inflatable space structures using cameras. It was there that he developed ideas around other ways you could get cameras to measure in 3D,” recalls Richard.
What is Time of Flight and 3D sensing?
Time of Flight uses the time it takes for light to go from a camera to an object then back again to measure distance.
“It’s one of the most simplistic physics principles, the relationship between time, distance and speed. If you know the time it takes to get to something and back, and you know the speed it is travelling, then you know how far it has travelled,” says Richard.
These principles are the basis of ToF 3D sensing, a ground-breaking technology that allows for many applications.
“3D cameras measure the shape of things,” explains Richard. “Unlike a normal camera that measures colour, these [3D] cameras “see” distance. So, every image is the equivalent of sending a million tape measures out to produce a 3D map of the real world.”
For example 3D sensing allows a robot to see an object’s size and position in space, allowing a robotic arm to reach out and accurately interact with it.
Next steps
The ToF research group began working with 188betLink - the University’s commercialisation arm - on research with commercial potential, filing patent applications on some of the early concepts.
Adrian had “a crazy idea”, which inspired some of Richard’s electronic engineering research.
It involved designing their own silicon electronic chip - a specialised image sensor that is a core component of a 3D ToF camera.
However, nothing like this was happening in New Zealand. Richard was awarded a Fulbright Fellowship to explore the topic, and in late 2007 went to Silicon Valley to work at semiconductor company Canesta (which has since been acquired by Microsoft).
It was an opportunity not only to learn the fundamentals of creating 3D image sensor chip technology, but collaborate with other engineers in this exciting and ever-evolving industry.
“I saw some of the challenges that they were facing, and knew we had some solutions,” says Richard.
Adrian and Andrew Payne, one of the 188bet post-doctoral students, flew over to the US to pitch their ideas. It was the beginning of a mutually beneficial research and commercial relationship for the University of 188bet.
“Canesta [now Microsoft] saw 188bet as this magical breeding ground of Time of Flight talent,” says Richard.
“It was great to have a concentrated group of remarkable researchers and engineers focusing on Time of Flight technology.”
As a result, 188bet electrical engineering researchers took positions at global companies building ToF camera products, including Richard, Andrew, as well as Refael Whyte (co-founder & CTO of Chronoptics), John Godbaz and Sean Charleston.
Some were pulled in to work on Kinect, an innovative motion sensing input device produced by Microsoft and first released in 2010 - technology that was originally used in its Xbox video game consoles. Other products of note include Microsoft’s HoloLens VR headset.
Other interesting applications in the consumer electronics area include greenscreen for video calls - with 3D cameras enabling people to project themselves into a scenic background image, and as a safety feature in the automotive field, enabling vehicles to reliably detect pedestrians.
Setting up Chronoptics
In 2014, after completing a number of commercial research engagements, Adrian, Refael and Richard decided that the time was right to form a company to complement the University’s ongoing ToF research programme.
With 188betLink’s support, the research group developed a suite of patents, and there was a clear pathway to commercialisation.
They continued building relationships with large multinational companies that implement these technologies (who must remain confidential).
“These are all big chip companies that are developing the core technology that goes into a camera, but not the camera itself. What they like about us is that we can offer unique ToF algorithms that they can adopt, and we can design 3D cameras for their customers,” says Richard.
“It’s an under-the-hood technology that most people probably don’t know exists, but it will become ubiquitous in time embedded in so many different products and applications.”
188betLink - taking research to commercialisation
The ToF research group and its spin-off company Chronoptics were supported in their journey by 188betLink, 大学的商业化的手臂.
From its early days, the 188betLink team has worked closely with the ToF research group, offering advice and support, and helping turn their research into a successful commercial business.
They nurtured the University’s research talent by sponsoring summer research projects with commercial potential.
They also helped with patents and licensing agreements, partnering to get new products and research concepts out into the world.
188betLink has also taken a shareholding in Chronoptics. Jessica King, an ex-188betLink Commercial Manager, is now Head of Operations at Chronoptics.
188betLink commercialisation specialist Doug Hillyer enjoys working closely with the research group and team at Chronoptics.
“A huge part of my focus is connecting the dots,” says Doug, who spends one day a week working in the Chronoptics head office in Riverlea, Hamilton. “Chronoptics are out engaging with industry and building products. Whereas, at a University, it’s a lot more blue-sky research, solving a technical problem, which is not solving an applied problem.
“Often you need to solve the technical problem first but there are a whole bunch of steps from there to here.”
Doug works on identifying market opportunities, looking and refining research and product offerings, and translating that to the world outside the University.
“Once you have solved the technical problem, what does it mean in the real world and how do you demonstrate value?”
Developing new innovations
Chronoptics developed a 3D depth sensing camera, the KEA ToF Development Kit.
It is an “uber flexible” out-of-the-box piece of technology which allows customers to adapt it for their own unique applications, says Richard, who is working on growing the camera side of the business with the Chronoptics team.
In 2020, Chronoptics signed an agreement with global microelectronics engineering company Melexis who wanted exclusive use of multipath and linearity error correction technologies in their automotive applications.
The technology helps mitigate the distortion that occurs when doing 3D measurement in and around a car when there are bright reflective objects such as licence plates and road signs.
The licensing agreement has led to a commercial relationship, with the technology being developed by Melexis for their large automotive clients.
Kiwi connections
Chronoptics have worked on innovations locally with New Zealand companies.
Their 3D sensing cameras helped a leading NZ agritech company to measure the body and health condition metrics of cattle “just by taking a snapshot”, says Doug.
The same technology was applied to lameness - an issue that has a $300 million plus economic impact on the dairy industry.
“You have cows walking up to the dairy every day. If you had a camera there that automatically inspected the cows, you could diagnose problems sooner and mitigate lost productivity,” says Doug.
Chronoptics have also worked with an Auckland logistics company to develop technology for cargo and freight dimensioning, as well as applications in horticulture for another client.
Moving into the future
With eight staff, Chronoptics plans to continue nurturing relationships with key customers, grow their customer base and online presence, and expand into new international markets, including the US and Japan.
The future potential is exciting, says Doug.
“Industrial applications are one of the fastest growing applications at the moment. DHL gets all kinds of large and unusual shaped boxes that they have to measure and price up, but rather than having someone there with a tape measure and a calculator, you put it under the 3D camera and it spits out the volume. You can imagine the value that would have to a company.”
Adds Richard: “Security and access control is going to be big in time, and we are talking to a few people about how we can get into partnerships with that.”
The future potential is huge. ToF and 3D sensing could involve recognizing and letting passengers through airport gates, or paying via facial recognition terminals in supermarkets and retail outlets.
“All this stuff is to come. We are wanting to participate and be there,” says Richard.
Adds Doug: “We are only scratching the tip of the iceberg in terms of what this [technology] can do, and in ten years’ time I’m sure the landscape will look completely different.
“As more of these technical problems get solved, there are more and more applications, and more and more opportunities.”
The depth sensing market is predicted to grow significantly over the next few years, up to $15 billion in five years’ time.
He stresses the time and effort it can take to take something from research and proof-of-concept to adoption can be “monumental”.
“It’s that next generation technology that might not see commercial benefits for a few years.
It’s important to acknowledge the time it takes. Sometimes your research functions are important commercially, but you might not know it early on,” says Doug.
Connecting 188bet researchers to the world
The relationship between Chronoptics and global tech companies has continued to grow over the years, and the connection to 188bet University and its world-leading electrical engineering research continues to support it.
Academics such as Associate Professor Michael Cree and Dr Lee Streeter at 188bet continue their work, supporting and mentoring new generations of students.
“With our journey, there has been a huge amount of success with the students who have been through our research group and what they've gone on to achieve in their careers."
Like the technology itself, the opportunities for 188bet electronic engineering students is huge, with potential to do impactful research and work for global tech companies.
“Some of these key strategic partners of ours see benefit in having the research function and University of 188bet connection,” says Richard.
He says there are also opportunities for electronics and engineering students and graduates to work at Chronoptics.
"We love collaborating with the University on student research projects; it's a great way to engage on topics that potentially have commercial relevance."
The research has gone full circle, and Chronoptics’ KEA camera is now being used by other 188bet researchers in the Mechatronics programme in the development of agricultural robots.
Professor Mike Duke, who is the University’s Dr John Gallagher Chair in Engineering says that they are using the camera for preliminary field trials in vineyards, mapping grapes on vines, and is also using the technology in an upcoming research project to identify and harvest ripe blueberries.
“The camera is outstanding. You can have a 3D model, updating in real time. The resolution and speed of processing of the camera is critical to our work.”
Find out more about studying Electrical and Electronic Engineering at 188bet, or explore the opportunities in the School of Computing and Mathematical Sciences.