Tier IV's Sensor Evaluation & Benchmarking service will start from an earlier timing in 2021 with a β version. The β version will be by invite only, offered first to our LiDAR partners and members of the Autoware Foundation. Depending on demand, we include other manufacturers in the β up to 10 companies in total. The first evaluation and benchmarkcing test requires a minimum of 5 participants.
As a company focusing on democratizing autonomous driving software and systems, and through involvement in the open-source Autoware software project and the Autoware foundation, Tier IV together with its subsidiaries and partners recognized the need for evaluation and comparison of different sensors for different Operational Design Domains (ODDs). From both the point of view of sensor manufacturers, and also consumers of such products, there is a lack of standardized testing to evaluate and compare sensors for autonomous driving tasks. This project focuses on LiDAR sensors, as a key technology for enabling autonomous driving.
Evaluation and comparison of different LiDAR sensors for autonomous driving potentially covers a wide array of testing in varied environments. It was recognized that standardized testing in adverse weather conditions is not only very difficult to achieve but also very valuable in the evaluation of a particular sensor. Therefore, Tier IV proposes an evaluation and benchmarking framework for performing comprehensive testing in adverse weather conditions, in a controlled environment.
The Sensor Evaluation and Benchmarking service allows you to confirm your product's performance in a variety of environments, and compare sensor performance to other manufacturers.
Your sensor will be tested in a controlled environment and results analysed in detail. The testing includes evaluation and benchmarking in the following categories:
The performance analysis and certification provided by Tier IV (as a third party) can be used in promotional material and campaigns.
Your R&D team can use the report for identifying areas for improvement, and allows benchmarking against other manufacturers.
Your Corporate team can use the report for any publication.
Q: If there is any damage to the product while doing the test, will it be compensated by Tier IV?
A: No, any damage or errors happened as a result of the test environment cannot be claimed to Tier IV.
Q: Is there any relationship with LIBRE by Nagoya University?
A: No, there is no relationship with the research activity by Nagoya University. This service is a commercial service and is completely separated from academic projects.
Q: Can any type of LiDAR be tested?
A: Yes, we accept all types of LiDAR. E.g. Mechanical Rotation, MEMS, Flash, etc.
Q: What is the criteria of the score from 1 to 5?
A: A sample report can be provided, explaining the criteria for each benchmark score.
Q: Is it possible to know the names of the other manufacturers' products in the benchmark?
A: Basically it is not possible to disclose the names, however, if the manufacturer had agreed in advance to disclose the name, then it will be included in the benchmark results.
Q: Is it possible to change from "Disclose model on the benchmark" to "Do not disclose model on the benchmark" after publication?
A: This is not possible if the request comes after publication. It is possible only if the notification is received 5 business days before publication.