Samsung has just been discovered to skew the benchmark tests of the Galaxy Note 3, but the Korean group is not the only company.
Through an in-depth analysis performed by AnandTech, we have learned that this is a very common practice among OEMs, in which manufacturers implement system optimization software to push to the maximum CPU performance during these tests.
According to report, the devices of all major Android OEMs — with few exceptions – falsify and have illegally optimized to provide the best results in major benchmark tests. This means that, whenever I run certain benchmark tests, the CPU of the device reaches its maximum speed, so as to increase special scores. Some manufacturers seem to apply similar tactics to the GPU.
Until now, such bogus practice had been discovered only on Samsung smartphones including Galaxy S4 and Galaxy Note 3. Other industry leaders are also adopting the same trick in particular ASUS, HTC and LG have illegally optimized for at least one test.
Here’s a table that shows all kinds of benchmarks run on various devices: The letter “Y” indicates that the product cheats these tests and the letter “N” indicates that the device is not optimized.
|3DM||AnTuTu||AndEBench||Basemark X||Geekbench 3||GFXB 2.7||Vellamo|
|ASUS Padfone Infinity||Qualcomm Snapdragon 800||N||Y||N||N||N||N||Y|
|HTC One||Qualcomm Snapdragon 600||Y||Y||N||N||N||Y||Y|
|HTC One mini||Qualcomm Snapdragon 400||Y||Y||N||N||N||Y||Y|
|LG G2||Qualcomm Snapdragon 800||N||Y||N||N||N||N||Y|
|Moto RAZR i||Intel Atom Z2460||N||N||N||N||N||N||N|
|Moto X||Qualcomm Snapdragon S4 Pro||N||N||N||N||N||N||N|
|Nexus 4||Qualcomm APQ8064||N||N||N||N||N||N||N|
|Nexus 7||Qualcomm Snapdragon 600||N||N||N||N||N||N||N|
|Samsung Galaxy S 4||Qualcomm Snapdragon 600||N||Y||Y||N||N||N||Y|
|Samsung Galaxy Note 3||Qualcomm Snapdragon 800||Y||Y||Y||Y||Y||N||Y|
|Samsung Galaxy Tab 3 10.1||Intel Atom Z2560||N||Y||Y||N||N||N||N|
|Samsung Galaxy Note 10.1 (2014 Edition)||Samsung Exynos 5420||Y(1.4)||Y(1.4)||Y(1.4)||Y(1.4)||Y(1.4)||N||Y(1.9)|
|NVIDIA Shield||Tegra 4||N||N||N||N||N||N||N|
So there are some devices, which don’t fake benchmark tests — that is the Motorola Razr, Moto X, Nexus 4, Nexus 7 and NVIDIA Shield — but the majority do it, and some products have been unethically optimized over the others. For example, LG G2 and ASUS PadFone Infinity score higher than Galaxy Note 3 (the “cheating mode” was active) when tested especially with AnTuTu and Vellamo.
“The hilarious part of all this is we’re still talking about small gains in performance. The impact on our CPU tests is 0 – 5%, and somewhere south of 10% on our GPU benchmarks as far as we can tell. I can’t stress enough that it would be far less painful for the OEMs to just stop this nonsense and instead demand better performance/power efficiency from their silicon vendors,” concludes AnandTech.
All these fake optimizations aren’t done by chip makers, but only by OEMs. Besides Apple, Google and Motorola are truthful and still respectable because they know the potential benefits of fake tests are very minimal.