Then, the ROM would alter the frequency in relation to an adjusted CPU load - our tools showed CPU load would drop to 0% regardless of obvious activity within the application, and the CPU would see a near-minimum frequency of 1.29GHz in the big cores and 0.98GHz in the little cores. Such application names were explicitly listed by their package IDs within the ROM in a manifest that specified the targets. So how does it work, and what's the difference? Last time around, OnePlus introduced changes to the behavior of their ROM whenever it detected a benchmark application was opened. We only know it targets the same packages. With the OnePlus 5, we see a different kind of cheating mechanism, but we cannot pinpoint whether this was consciously introduced by the same developers who added it the first time around. It made sense to us at the time and comments from OnePlus representatives made to XDA-Developers added credibility to our theory. We attributed these changes to the recent merger of the then-disparate OxygenOS and HydrogenOS developer teams, and the underlying codebase of OxygenOS which was now to be shared with HydrogenOS, though this speculation is yet to be confirmed. Last January, our report unearthed a cheating mechanism found in OxygenOS Beta builds and in the shipping software of the OnePlus 3T. All temperatures were measured using a FLIR C2 Compact with each endurance run beginning at an outer temperature of 28.5☌ | 83.3☏. As for testing, the ROM had minimal background processes with no third-party applications and running Airplane Mode where applicable CPU frequencies were logged only to determine the extent of the cheating and not in the tests that produced scores for this article. It did clue me into the fact that OnePlus was referring to benchmark packages by name in their ROM. OnePlus forwarded reviewers instructions to enable the ability to download benchmark applications off the Play Store, and presumably this was done so that there would be no benchmark score leaks ahead of time. While we do not believe this feature article necessarily should alter your perception of the hardware itself, it is right for it to nudge your opinion of the company given it is their second transgression.Īll scores on this article have been obtained on a OnePlus 5 review unit running OxygenOS version 4.5.0 (A5000_22_170603) this is a pre-production unit, and it was originally loaded with pre-production software which received an OTA to the version named above. Finally, we will be compartmentalizing this report from our overall judgment of the device itself, because we are confident the culprit code will be removed from consumer builds following this report and our conversations with OnePlus representatives. We also will not provide a full performance analysis involving all of our included tests as many of our preferred benchmarks are affected by the cheating mechanism. What is worse is that, this time around, the cheating mechanism is blatant and aimed at maximizing performance, unlike last time which did not increase scores by much on average, but did reduce variance and thermal throttling, as we found.īefore we jump into the details, I would like to state that we are disappointed in the company for once more resorting to these practices. As a result, every OnePlus 5 review citing benchmark scores as an accolade of the phone’s success is misleading both writers and readers, and performance analyses based on synthetic benchmarks are invalidated. This is an inexcusable move, because it is ultimately an attempt to mislead not just customers, but taint the work of reviewers and journalists with misleading data that most are not able to vet or verify. Unfortunately, it is almost certain that every single review of the OnePlus 5 that contains a benchmark is using misleading results, as OnePlus provided reviewers a device that cheats on benchmarks. While no customers have a device in their hands (it just launched after all), we have learned about OnePlus’ new benchmark cheating mechanism through our review unit, which we received about ten days ago before the day the embargo breaks and reviewers are allowed to report on the device. Today, we sadly must follow-up on our accusations as the company has once more been inappropriately manipulating benchmark scores in the OnePlus 5. Earlier this year, we published a report that denounced OnePlus (and other companies) for their improper behavior in regards to benchmark manipulation on newer builds of OxygenOS.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |