
By Jeremy Geelan | Article Rating: |
|
June 15, 2004 12:00 AM EDT | Reads: |
246,885 |
Keith Lea writes of the benchmark, on his results page, "I was sick of hearing people say Java was slow, when I know it's pretty fast, so I took the benchmark code for C++ and Java from the now outdated Great Computer Language Shootout and ran the tests myself."
Lea used G++ (GCC) 3.3.1 20030930 (with glibc 2.3.2-98) for the C++, with the -O2
flag (for both i386
and i686
). He compiled the Java code normally with the Sun Java 1.4.2_01 compiler, and ran it with the Sun 1.4.2_01 JVM. He ran the tests on Red Hat Linux 9 / Fedora Test1 with the 2.4.20-20.9 kernel on a T30 laptop. The laptop "has a Pentium 4 mobile chip, 512MB of memory, a sort of slow disk," he notes.
The results he got were that Java is significantly faster than optimized C++ in many cases.
"They also show that no one should ever run the client JVM when given the choice," Lea adds. ("Everyone has the choice," he says. To run the server VM, see instructions in the Using the Server JVM section below.)
JDJ has agreed to post online anyone else's results as long as they use Java 1.4.2 or higher and any version of GCC that produces faster or equivalent code than the 3.3.1 I used. We encourage you to download the source and/or the binaries and perform the tests yourself, with your favorite compiler and on your favorite platform.
Lea's Data and Results
- Data table and graph (also see above)
- Console run log
- Source:
- Binaries
- Download all source code and binaries: .bz2, (69KB) .zip (189KB)
JVM startup time was included in these results. "That means even with JVM startup time, Java is still faster than C++ in many of these tests," says Lea.
Some of the C++ tests would not compile. "I've never been very good at decoding GCC's error messages," he admits, "so if I couldn't fix a test with a trivial modification, I didn't include it in my benchmarks."
Lea also modified one of the tests, the string concatenation test for Java.
"The test was creating a new StringBuffer in each iteration of the loop, which was just silly," he explains. "I updated the code to use a single StringBuffer and appending to it inside the loop."
(The updated tests at the original shootout use this new method.)
"Java lost this benchmark even with the modifications," Lea declares. "So if anyone wants to accuse me of biasing the results, they're going to have to try harder."
Several versions of some of the C++ tests (like matrix
) were present in the original shootout source, he continues.
"I used the versions without numbers in them, like matrix.g++
instead of matrix.g++2
. I don't know which of these were used in the original benchmarks, but from my quick experimenting, the numberless ones generally ran faster than their numbered counterparts."
"Looking at them again," Lea says, "matrix.g++3
runs faster than the matrix.g++
that I use. However, it still runs slower than the Java version, so I don't plan to modify the graph/data unless someone asks me to, since getting that graph in the first place was sort of a pain.)"
He continues: "I've been told that the C++ code for the Method Call benchmark returns by value while the Java code returns by reference, and that modifying the C++ code to pass a pointer makes that benchmark faster. However, even with the modification, the C++ version still runs slower than the Java version."
Lea ran th Java and the C++ tests to "warm up" (both the Java and C++ tests got faster after he ran them a few times).
"I've been told that these tests are invalid because they were run with GCC," he concedes, adding: "I have seen both benchmarks that show GCC producing faster code than Visual Studio's VC++ compiler, and benchmarks showing the opposite. If I update the benchmarks with another compiler added, it will be the Intel C++ Compiler, which I'm pretty sure produces faster code than VC++."
Lea says he's been accused of biasing the results by using the -O2
option for GCC, "supposedly because -O2 optimizes for space, thus slowing down the benchmark," he explains.
But this is not what -O2 does, he points out, referring to the GCC -O documentation:
JVM startup time was included in these results. "That means even with JVM startup time, Java is still faster than C++ in many of these tests," says Lea.
Some of the C++ tests would not compile. "I've never been very good at decoding GCC's error messages," he admits, "so if I couldn't fix a test with a trivial modification, I didn't include it in my benchmarks."
Lea also modified one of the tests, the string concatenation test for Java.
"The test was creating a new StringBuffer
in each iteration of the loop, which was just silly," he explains. "I updated the code to use a single StringBuffer and appending to it inside the loop."
(The updated tests at the original shootout use this new method.)
"Java lost this benchmark even with the modifications," Lea declares. "So if anyone wants to accuse me of biasing the results, they're going to have to try harder."
Several versions of some of the C++ tests (like matrix
) were present in the original shootout source, he continues.
"I used the versions without numbers in them, like matrix.g++
instead of matrix.g++2
. I don't know which of these were used in the original benchmarks, but from my quick experimenting, the numberless ones generally ran faster than their numbered counterparts."
"Looking at them again," Lea says, "matrix.g++3
runs faster than the matrix.g++
that I use. However, it still runs slower than the Java version, so I don't plan to modify the graph/data unless someone asks me to, since getting that graph in the first place was sort of a pain.)"
He continues: "I've been told that the C++ code for the Method Call benchmark returns by value while the Java code returns by reference, and that modifying the C++ code to pass a pointer makes that benchmark faster. However, even with the modification, the C++ version still runs slower than the Java version."
Lea ran the tests many times before running the "official" recorded set of tests, so there was plenty of time for both Java and the C++ tests to "warm up" (both the Java and C++ tests got faster after he ran them a few times).
"I've been told that these tests are invalid because they were run with GCC," he concedes, adding: "I have seen both benchmarks that show GCC producing faster code than Visual Studio's VC++ compiler, and benchmarks showing the opposite. If I update the benchmarks with another compiler added, it will be the Intel C++ Compiler, which I'm pretty sure produces faster code than VC++."
Lea says he's been accused of biasing the results by using the -O2
option for GCC, "supposedly because -O2 optimizes for space, thus slowing down the benchmark," he explains.
But this is not what -O2 does, he points out, referring to the GCC -O documentation:
-O2: Optimize even more. GCC performs nearly all supported optimizations that do not involve a space-speed tradeoff. The compiler does not perform loop unrolling or function inlining when you specify -O2. As compared to -O, this option increases both compilation time and the performance of the generated code.
"On the other hand, -O3 performs space-speed tradeoffs, and -O performs fewer optimizations. Thus, for these tests, I think O2 was the best choice," Lea concludes.
"I don't have an automated means of building and benchmarking these things (and the scripts that came with the original shootout didn't run for me)," he continues. "I really do want people to test it on their own machines, but it's going to take some work, I guess."
Lea compiled the C++ code with:
g++ [test].cpp -O2 -march=i386 -o [test]-386
g++ [test].cpp -O2 -march=i686 -o [test]-686
and the Java code with:
javac [test].java
To see how he ran the binaries, see the run log. You can download the source code he used in either .bz2 or .zip format.
Using the Server JVM
Every form of Sun's Java runtime comes with both the "client VM" and the "server VM."
"Unfortunately, Java applications and applets run by default in the client VM," Lea observes. "The Server VM is much faster than the Client VM, but it has the downside of taking around 10% longer to start up, and it uses more memory."
Lea explains the two ways to run Java applications with the server VM as follows
- When launching a Java application from the command line, use
java -server [arguments...]
instead ofjava [arguments...]
. For example, usejava -server -jar beanshell.jar
. - Modify the
jvm.cfg
file in your Java installation. (It's a text file, so you can use Notepad or Emacs to edit it.) This is located in C:\Program Files\Java\j2reXXX\lib\i386\ on Windows, /usr/java/j2reXXX/lib/i386/ on Linux. You will see two lines:-client KNOWN -server KNOWN
-server KNOWN -client KNOWN
-client
argument.
He can be contacted at
Every form of Sun's Java runtime comes with both the "client VM" and the "server VM."
"Unfortunately, Java applications and applets run by default in the client VM," Lea observes. "The Server VM is much faster than the Client VM, but it has the downside of taking around 10% longer to start up, and it uses more memory."
Lea explains the two ways to run Java applications with the server VM as follows
- When launching a Java application from the command line, use
java -server [arguments...]
instead ofjava [arguments...]
. For example, usejava -server -jar beanshell.jar
. - Modify the
jvm.cfg
file in your Java installation. (It's a text file, so you can use Notepad or Emacs to edit it.) This is located in C:\Program Files\Java\j2reXXX\lib\i386\ on Windows, /usr/java/j2reXXX/lib/i386/ on Linux. You will see two lines:-client KNOWN -server KNOWN
-server KNOWN -client KNOWN
-client
argument.
He can be contacted at [email protected].
Links
- Java vs. C benchmark - the benchmark is in Microsoft Word format
- FreeTTS performance case study - at Sun Research
- Linux Number Crunching - used 1.4.1 VM, got very different results than Lea did, maybe due to floating point optimizations added in 1.4.2
- SNAP matrix benchmark - Java vs. C++ for matrix multiplication
- Nine-Language Performance Round-Up: math and file I/O performance among nine languages
- Performance comparison C++, C# and Java - similar results as Lea's, mostly
- Java vs. C - for bubble sort, IBM's 1.3.0 client JVM comes out on top over GCC with full optimizations
Published June 15, 2004 Reads 246,885
Copyright © 2004 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Jeremy Geelan
Jeremy Geelan is Chairman & CEO of the 21st Century Internet Group, Inc. and an Executive Academy Member of the International Academy of Digital Arts & Sciences. Formerly he was President & COO at Cloud Expo, Inc. and Conference Chair of the worldwide Cloud Expo series. He appears regularly at conferences and trade shows, speaking to technology audiences across six continents. You can follow him on twitter: @jg21.
![]() |
Daedalus 06/09/08 10:10:21 AM EDT | |||
This dude has no idea whatsoever about C++, probably he doesn't even realise how flawed his benchmarks are, and he believes his senseless results. If anyone cares, -O2 is indeed a very wrong choice. So is allocating objects through operator new, so is passing objects by value, so are a lot of other things, which prove that he is either trying to rip us off straight away, or too stupid to realise that he cannot make a proper benchmark without knowing both languages. |
![]() |
jmerd 11/23/07 12:23:12 PM EST | |||
This is absolutely funny. You do java, do not even say "performance" - as long as you do not even know what asynchronous io is (if you are lucky you are now at the non-blocking stage). Also - the JVM is a C/C++ program - meaning: if Java is faster than C then C is as fast as Java (paradox!). Go on - play with your toys. |
![]() |
aaa 08/01/05 12:32:28 PM EDT | |||
I only registered because the benchmark is totally lame. Shame on you that wrote the benchmark and article! It's a completely bogus article, for many reasons: a) you use new-delete in C++ where it is not needed b) you do not use loop unrolling for C++(as you blatantly admit) I suspect you do not specify inlining, do you? There is no excuse for your lame article. The results are funny, especially the method call test that Java seems 10 times faster in calling methods than C++, whereas the actual slowness of C++ is the new/delete cycle! Ask yourself this: is it possible for a Java method call to be 10x times faster than compiled C++? I've tried the method call test by removing the new-delete and added time measurement inside the program, and C++ is faster than the server VM 1.4.2. I did not look any further. You are a joke, and a biased one. Oh, and Java is never gonna get as fast as C++ simply because Java runs in a JVM: there is machine code (written in C++ - oh the irony) that controls the execution of bytecode. |
![]() |
Mike 06/09/05 07:25:35 PM EDT | |||
I find the numbers reported on shootout incorrect. They should fix their test. Run the algorithm, whatever it may be in a method, and from main, call it a few times, say, 5 times. Have main measure how many milliseconds the called method takes, and print it to the screen. Run it with java -server as usual. This fixes 2 problems with shootout: I'm outraged out shootout has created a completely false view and made Java look worse than C and C#. |
![]() |
Isaac Gouy 11/09/04 02:40:15 PM EST | |||
>"outdated Great Computer Language Shootout (Fall 2001)" Shootout has been revived |
![]() |
TheSpider 07/06/04 03:12:28 AM EDT | |||
Larry, Very much appreciate your comments. Would like to connect via email to discuss possibilities. Perhaps through the author or via purpose created Yahoo email account. Please advise if interested and if so preferred method of contact. |
![]() |
larry 07/05/04 12:41:49 PM EDT | |||
Much of that article is out of date w.r.t java. Perhaps he can re-look at it again and update it. no point in debating old point that are not relevant anymore |
![]() |
héhé 07/02/04 03:01:11 AM EDT | |||
this reflect what I meet daily in Java compare to C++ |
![]() |
icube 06/30/04 09:46:13 AM EDT | |||
shootout has a great interest but not between C++ and java but between ocaml and java .... :-)))) java .net .... are marketing tools they all lay on their own performance ... only OCAML can be put in a nuclear plant (see the java license ;-) ! why ? no compromise ... with complexity and strenght ... strongly typed when java start at jdk 1.5 ( ~ with 20 years) but with bad ;-) tools many people can do business ... write books .... java performance tuning (a good book also for java world!) and provide services ... version of jVm etc ... java and .net are arifacts of the object marketing big bazard ...put programmers community attention on OCAML and let programmers discover concepts and features, performance of vm ;-),let them, develop OCaml stack and one again check performance with J2EE or .NET ? ;-) who wins ? In shootout performance, OCAML is closer to C++ and C , overall winner of minimal line of code ... ,overall winner in platform independant. Why Microsoft R&D investigate on F# an OCAML sub language for its CLR layer ;-) and why OCAML is the best programming languge of MIT and INRIA(French R&D, its fonder) teams ? be aware & wake up !!! by a java tuning specialist ... ocamly converted @;-) icube |
![]() |
MJD 06/27/04 10:37:49 PM EDT | |||
Concerning C++ issuing deletes and Java waiting for GC to run. If you are done with an object in java, you too can set the object to null. jc = null acts as a hint to the garbage collector that the object space is ready for garbage collection, and it also will probably re-catagorize the object as a shortlived object and be handled by a different garbage collector, a faster one. It does look like Java is a true replacement for beginner and intermediate C++ programmers. Especially those that lose track of their objects and never actually issue those deletes, and those that don''t use code profiling. So, will experience C++ programmers write systems that beat java in the real world. Most times there isn''t the time or money in the budget to do so. So, in the real world the answer is no. I''ve seen Microsoft C++ shops ship the debug build instead of the release build because the release build leaked memory all over the place. And it was the Microsoft libraries that were leaking memory. In my experience the language VB, C++ or Java was never the issue in business applications. In Business Apps the Database is always the bottleneck. Anyway, a pretty good thread. |
![]() |
Gomes 06/25/04 01:54:32 PM EDT | |||
I compiled the Fibonacci program in Visual C++.Net (Release mode) 22 seconds (Athlon XP1800 256 RAM) ======== Conclusion: Is Visual C++.Net Faster Than JVM(Linux)? Is Athlon XP 1800 better than Pentium(???Frequency???) MObile 512MB of memory? Above the results of the author in Pentium MObile 512MB of memory... ======== [keith@leak bench]$ time cpp/fibo 45 real 0m44.042s real 0m50.625s [keith@leak bench]$ time java -server -cp java fibo 45 real 0m33.842s [keith@leak bench]$ time java -cp java fibo 45 real 0m50.066s ------------------------------ |
![]() |
jose 06/24/04 06:20:46 AM EDT | |||
5 - 6 years??!!! What, did they hire monkeys? To think of whats things I could write in that time. Terrible... Anyways, yes, there''s no doubt that Java will be more effective on certain levels and the reason why is because Java was has specific areas covered quite well, although I will disagree to an extent about the wonders of LDAP. But alas this is Sun we are talking about and this is their bread and butter. I think you are forgetting that C++ is just a language. At the end of the day, its about what technologies you decide to mix together. Visual Basic, for example, was Microsoft''s attempt at doing something quite similar to that (but we all know VB is crap and so will hesitantly use it). In that case, I''d like to share my experience. When I once was at univeristy a while back, we had to write a object tracker using digital imaging principles, that involved taking input from a camera source and distinguishing cars from other objects and tracking their velocity and travel. I did mine, naturally, in C++, while a class mate of mine did it in Java. I wrote mine in half the time as he did and it was more reliable and faster. Does this prove that an implementation was much better in C++ ? Not necessarily, it could also prove that he simply was a poor Java developer, or that he had more trouble picking out the correct methods and technologies to use for his project. Apples and oranges. |
![]() |
Yakov Fain 06/23/04 10:51:23 PM EDT | |||
Most of the postings here compare particular Java and C++ features. If you go this route, C++ will probably beat Java. But let me share my experience with you. A couple of years ago I was involved with creation of a real-time trading system in Java for a major brokerage house. At that time, a system written in C++ was in production for 5 or 6 years. For the new system we used J2EE with EJB, JMS (MQSeries) and LDAP servers. When the new system went into production, it was much faster that the old one and more stable. And let me tell you, the old C++ system was not poorly written! Once in a while C++ developers would say something like ?we are using memcpy in our system for fast copying of memory blocks, and your Java does not have it?. So what? The final product with bean pools, multithreading and asynchronous messaging was performing a lot better than a product written in C++. Please keep in mind, that we did not have to hire C++ gurus to deal with multithreading ? Java application servers did it for us! You may say that I am comparing apples and oranges. True ? I am comparing J2EE with C++. Unfortunately I did not have a chance yet to re-write a .Net system in J2EE ? |
![]() |
larry 06/23/04 05:03:10 PM EDT | |||
Hi Spider, I hadn''t responded because this was slightly off topic and the answer can be the subject of several books. ;) Anyway, with respect to your items, here is a quick response 1. This is the classic architecural dilemma. I would suggest that you review a number of architectural patterns. There is a good book "Pattern Oriented Software Architecture". For example, the microkernel pattern and the layering pattern 2. A java VM will solve much of this problem for you. However there is a reverse memory leak issue. Objects remain uncollected so long as there is a strong reference to them. Current practices for modern VMs state that you do not pool small objects. Pooling is primarily for those java objects backed by native resources. You must be careful about creating too many objects however as you will pay for it in gc later. I would suggest that you use jvmstat 2.0 for j2se1.4.2 and above and jvmstat 1.1 for 1.4.1 and lower from the cool stuff web site on java.net 3. I would recommend that you manage your inheritance hierarchy so that you do not have an excessive amount of inheritence levels. Refactoring will help there. Modern OO practice de-emphasize inheritance over composition. 4. You will pay for security through performance. Rather than relying on applets solely, take a look at java net launch protocol as a way of deploying applications via web browser. This will get around various plugin issues. 5. In this case the new io package is the way to go (java.nio). Nio will allow you to achieve this. 6. In this case, you should invest in top notch java ui developers. Summing up, plan your architecture very carefully, use patterns to decouple layers so that you can maintin it easily and separte concerns. Utilize the right sort of java technology for the right purpose. |
![]() |
TheSpider 06/23/04 02:57:20 PM EDT | |||
Larry (or others), I''m still very interested in those "architectural" issues you noted. It would be great if you could be more specific. My main purpose for possibly selecting Java for our major code base conversion is for the following reasons: 1) A nice tradeoff between being "low level" enough to provide lots of flexibility vs. human efficiency and extensibility. We will need to get very low level at times to do some trick things but we want to code at as high a level of abstraction as we can. 2) Virtual elimination of memory management related leakage issues and pointer induced errors (this is a BIG deal). The biggest drawback to C or C++ is the huge issue with these kinds of problems. 3) Complete OO with adequate inheritance capability. Due to the dynamic nature of OO w/Java, we suspect this will be one of the areas we need to pay close attention to regarding performance. 4) Improved security for apps in a distributed environment. For wide area deployment, lightweight, secured applets talking over encrypted channels to heavy backend systems seems like a great way to reduce TCO. 5) High degree of near native connectivity over a distributed environment and amoung widely disparate devices. This kind of flexibility is not found in any other environment. 6) Ability to produce a powerful and flexibile user interface that is state-of-the-art. One of the reasons our software and service are selected is because we take a lot of time and effort to package sophisticated behavior into a form that is extremely efficient and natural to use. All the above being said, our first experience a few years ago was not pretty. The code was small and compact and highly functional but it was really, really, really slow. After investing several man-months in coding and testing (and investigating other toolsets and utilities) we finally scrapped the project until such time as Java or another tool could manage the workload while still keeping us efficient and logically correct. Those last two criteria are essential. Efficiency of human effort when porting a system with hundreds of RDBMS tables, hundreds of apps and hundreds of thousands of lines of code is a BIG project (at least for us). Keeping it logically verifiable is absolutely essential as this is a business suite that deals with large amounts of assets and cash flows. So I reiterate: What are the really important architectural issues that we should watch out for? What are the essential do''s and don''ts that can make a break a project of this size and nature? |
![]() |
Rockwell Chad 06/23/04 08:36:13 AM EDT | |||
Not all JRE installations come with the server vm ready to go. Please read this article to enable it for Win32: |
![]() |
MamanE 06/22/04 07:50:10 AM EDT | |||
I expect lower performance of the server VM on an Athlon than the server VM on a P4. What I don''t expect to see is the server running slower than the client on an Athlon, so I still think it could be a performance bug in the server. |
![]() |
Daniele Paccaloni 06/22/04 06:57:44 AM EDT | |||
MamanE, Also, note that when we say "as fast as the ASM version" we are actually comparing the SSE2 JIT code to my hand-coded ASM x87 code. You''d enjoy FFFF even more if you run it on a fast GPU (like ATI''s R300).. you can reach over 1 GigaIters per second ;) |
![]() |
MamanE 06/22/04 05:14:35 AM EDT | |||
"Also, would someone please run the mandel test on a P3 or Athlon-XP (thus forcing the JIT to use the FPU) ?" The numbers I gave earlier (where the client ran about as fast as the C version and the server strangely enough a bit slower) are done on an AthlonXP. The numbers where the server runs as fast as the ASM version was done on a P4. Now I don''t know if something is screwed on my Athlon PC or if there is a performance bug in the server VM related to an Athlon, but I''ll check that out. |
![]() |
MamanE 06/22/04 05:09:00 AM EDT | |||
Daniele''s numbers on the server VM are definitely right. I tested here and on the server VM it''s as fast as the ASM version and almost 3.5 times as fast as the C version, and I checked the results too. Quite amazing if you ask me. |
![]() |
Daniele Paccaloni 06/22/04 04:38:29 AM EDT | |||
Looking again at the mandelbrot src, I realized that the results are indeed used, so that the server JVM could not be optimizing by skipping any op. To see it more clearly, I did break into the JIT code with a debugger and saw that the JIT: 1) Arranged a 4x loop unroll. Now, that''s why the C code is slower: the C compiler is actually using default FPU (x87) code, so that.. the comparison is not fair ;) Of course this scores a point in favour of the JIT. Anyway, you can get the src on the same SourceForge thread (just scroll down to the new version): http://sourceforge.net/forum/forum.php?thread_id=1096782&forum_id=168353 I wonder if some gcc-guru could make it run as fast as the JIT version (porting the src to C is straight). Also, would someone please run the mandel test on a P3 or Athlon-XP (thus forcing the JIT to use the FPU) ? Let me know ! |
![]() |
MamanE 06/22/04 03:44:32 AM EDT | |||
"We can think jvm itself as a good c/c++ library." If a hammer is the only tool you got, every problem looks like a nail. |
![]() |
Eric Ma 06/21/04 08:13:53 PM EDT | |||
Hi Larry, Well call java bytecode "p-code" is inappropriate, bytecode is not machine code(bytecode->IR->machine code). Although a bytecode op, say "store", can be mapped directly to machine codes "push...;mov...;mov ...;...", jvm has to do that extra work on the fly. Also jvm need to map all code&data in bytecode into machine code&data address to be understood by cpu(also consider the dynamic nature of java class loading). We can think jvm itself as a good c/c++ library. In theory, code wrote to run directly should be faster than code built on this library. Jvm does be a good runtime(or library) for us. I''m discussing perf only, not which language is better. Language should be chosen per the task. |
![]() |
MamanE 06/21/04 07:56:20 PM EDT | |||
Hmmm, I''m getting the feeling those strangely good server numbers might be right after all and that something is screwed with my java installs. Somebody else had a run at it with a version which outputs results and on his machine, the server did the test about 3.5 times as fast! |
![]() |
MamanE 06/21/04 07:21:23 PM EDT | |||
That''s very interesting, larry. I''ve heard about the shared VM before and I''m sure it could well be the next big thing. |
![]() |
larry 06/21/04 05:49:54 PM EDT | |||
Well, the VM will definitely take a lot more. Using server will use more than client. The next big boost of VM technology will be the shared VM. There is a project at sun labs called the barcelona project The purpose of this project is to allow the sharing of a single VM instance between applications. So if you pay the cost once for VM instatiation and each additional instance uses very small amounts of memory. Part of this work is found in the Isolate JSR 121 (http://www.jcp.org/en/jsr/detail?id=121) Greg (Grzegorz Czajkowski) has done some very interesting work in reducing the footprint of applications using a VM. |
![]() |
MamanE 06/21/04 04:11:07 PM EDT | |||
Yeah, well this is still just one little test case and java still uses more memory than C++, although not as much as many people believe. The memory usage numbers can be quite deceiving if you don''t know anything about how the GC works and how to tune the VM using runtime options. |
![]() |
larry 06/21/04 03:44:11 PM EDT | |||
The interesting thing about all of this is that all the VM research is starting to pay off and VMs are actually competitive with native languages (grudges and denials not withstanding ;) ). The release of the JDK is actually several different teams, each quite independent of each other. Until the final release of 1.5 called FCS, I would expect the actual VM internals to fluctuate. So today client is faster, tomorrow ... |
![]() |
MamanE 06/21/04 03:20:56 PM EDT | |||
Yeah, you''re right. BTW I just tested the mandelbrot using 1.5.0 beta 2 on the client VM and I got this: A tiny little fraction faster than the C version :-) |
![]() |
larry 06/21/04 02:48:52 PM EDT | |||
Yes, you can get much of Sun''s implementation but the same is not true for Bea JRocket or the IBM impl. Those are well guarded. |
![]() |
MamanE 06/21/04 02:44:50 PM EDT | |||
"...using a variety of techniques all jealously guarded by VM manufacturers. ;)" Well, you *can* download the sources of Sun''s HotSpot compiler on their site ;-) |
![]() |
MamanE 06/21/04 02:35:49 PM EDT | |||
Although I might want to add, this applies for the server VM. I haven''t seen the client VM optimize a whole benchmark away yet. |
![]() |
MamanE 06/21/04 02:13:42 PM EDT | |||
" If this is the case, this could render synthetic benchmarks almost useless. Please comment." Yes you are right if you don''t use the results. Rule #1 for benchmarking java programs is to actually use the results or else the whole thing might get optimized away. Like I said, just printing the result does the trick. |
![]() |
MamanE 06/21/04 02:04:11 PM EDT | |||
I got for the java version: and the C version: This was with the slightly altered version where I made the code not static. The algorithms themselves are unchanged. I think that''s pretty good. I got strange server results as well. I''m almost 100% sure it''s because the whole thing gets optimized away since the results are not used so you should probably disregard that test. Printing the results will probably do the trick (and to be fair, print it outside the timed loop). |
![]() |
larry 06/21/04 01:27:57 PM EDT | |||
Daniele, It is possible. With the server switch, you select the most aggressive runtime optimizations Here is a short description of what happens. Where the VM loads a class, it creates a set of C++ objects representing each of the objects and classes of your application in the VM. Certain objects (such as threads, etc) have a special meaning to the VM and get their own specialized objects. There are not many of these however and they can be counted on 2 hands. Most objects however are described by the same class of C++ object. Once this C++ object is created it, it replaces each of the java byte codes by a corresponding sequence of native opcodes. This is still considered interpreted mode. The VM runs through much of this code during normal operation and creates a profile structure representing the runtime operation of the code. Once it has enough information, it compiles the "hotspots". Runtime optimization involves scanning through that section of the hotspot and optimizing the native op codes into tighter optimizations using a variety of techniques all jealously guarded by VM manufacturers. ;) |
![]() |
Daniele Paccaloni 06/21/04 12:49:56 PM EDT | |||
Sorry, the "correct" code is not "correct" :( The "for (int y..." is messed up. Please get the source from here: http://sourceforge.net/forum/forum.php?thread_id=1096782&forum_id=168353 |
![]() |
Daniele's code: 06/21/04 12:43:50 PM EDT | |||
Here's Daniele's code correctly displayed: public class JMandel { Or get the source from his SourceForge page: "http://sourceforge.net/forum/forum.php?thread_id=1096782&forum_id=168353" |
![]() |
Daniele Paccaloni 06/21/04 12:34:22 PM EDT | |||
I used the client VM. Runtime ms=3157 133.47453436807095 MegaIters per second This seems too fast to me since it equals the throughput of the hand-coded x87 Assembly code.. I suspect that the server JVM is skipping some processing if it "sees" that the results are not used outside of the method. But who knows ? I''m no Java guru so that I cannot say for sure. If this is the case, this could render synthetic benchmarks almost useless. Anyway I try to paste again the source code below, but it seems that the input form is messing up the text. public class JMandel { static int[] rowBuffer_ = new int[500]; static long msStart_; public static void main(String[] args) { |
![]() |
MamanE 06/21/04 12:28:19 PM EDT | |||
"Neither Java, nor C++ are the "panacea". A wise developer always chooses the proper language (or language-mix) suitable for the target solution." I couldn''t have said it better :-) |
![]() |
MamanE 06/21/04 09:09:19 AM EDT | |||
Hmmm it seems that some code got lost while copy-pasting the code into this board, so it''s kinda hard now to give it a try myself :-/ |
![]() |
MamanE 06/21/04 08:59:20 AM EDT | |||
I have a few remarks/questions: |
![]() |
Daniele Paccaloni 06/21/04 08:49:10 AM EDT | |||
I have arranged a little raw mandelbrot set synthetic bench to test floating point performance. As for Java I get: Runtime ms=10953 38.47 MegaIters per second Below are the scores of the original program (www.sf.net/projects/ffff) on the same P4 3.4 GHz. FFFF v3.2.1 BENCHMARK (Using 1 CPU, no render) SSE benchmark (32 bit float precision): SSE2 benchmark: FPU ASM benchmark: FPU C benchmark: [GPU benchmarks stripped] Here''s the Java code, it''s ugly but I have no time to make it better ;) public class JMandel { static long msStart_; public static void main(String[] args) { } |
![]() |
MamanE 06/21/04 07:24:51 AM EDT | |||
"This is an interesting point. Please consider that in games like Quake3 the real bottleneck is the graphics card. Even if the bottleneck was the CPU (i.e. Java/C++), I''''d prefer an 800fps C++ implementation instead of a 400fps Java implementation just because the C++ one will run at 70fps vsynced without eating up your notebook''''s battery." You''re quite right about the bottleneck not being the CPU. The point however is not whichever version you would prefer (battery life is of no concern at all to the general q3 gamer), the point is that java is perfectly suitable for making games which was a respond to another post saying you can''t do it. And consider that this particular quake3 clone was made in a fraction of the time of the real thing, by only one developer. |
![]() |
Daniele Paccaloni 06/21/04 06:56:57 AM EDT | |||
MamanE commented on 20 June 2004: You are right. But a faster VM will be able to do deeper runtime optimizations ;) MamanE commented on 20 June 2004: This is an interesting point. larry commented on 20 June 2004: While the first statment is "fanatic", they were quite right on the second statement. Neither Java, nor C++ are the "panacea". |
![]() |
MamanE 06/21/04 04:57:03 AM EDT | |||
"This is what I was trying to say. The difference between Java and C++ is not nearly as great as the difference between Assembly and C (and furthermore C++) to make the comparison at all. It''s more like making the comparison between C and C++, because each are alike in many ways and one just being a slight extension of the other." I have to disagree here. |
![]() |
M. Lipp 06/21/04 04:27:54 AM EDT | |||
This is the really one of the strangest collections of feedbacks that I have ever seen. Some are just silly (e.g. about Java being unstable, this is simply not true nowadays -- unless you try to optimize by using the native interface ;-)), but most interesting are the ones about the C++ code not being optimal or some compiler switches not being set. I haven''t used C++ for about 7 years now after having used it intensively for 7 years before. From a practical point of view: if C++ is faster only if developers know things like "come on, everybody knows that using XYZ is slow and you should use ABC", then this is of no relevance. Because the average developer DOES NOT KNOW. The average developer hardly really understands templates. You can make him use "StringBuffer" instead of "String" (although when profiling real world Java programs this has really never proven to be the issue), but you can not expect him to really understand what a compiler does. What do you think: how many average programmers out there have read Ellis/Stoustrup and understood at least most of it? So, of course, C++ offers you many possibilities to optimize your code. Eventually you can even insert assembler. But what counts is what is written at the initial implementation, because usually nowadays there is no time or budget to go over the complete code and optimize. - Michael |
![]() |
Lars Langhans 06/21/04 03:12:47 AM EDT | |||
IMHO the compiler options you used by g++ are not optimized. I checked only ackermann and fibo test and rebuild it on my own pc with follows options: (Tested on Linux 2.4.26/Gentoo gcc-3.3.3 glibc 2.3.3 Pentium4 1.8GHz) JM2C, |
![]() |
Jose 06/20/04 06:53:32 PM EDT | |||
"I remember getting into C in the early 80s. I had lots of people tell me C was crap and the only real way to do work was to do it in assembly." This is what I was trying to say. The difference between Java and C++ is not nearly as great as the difference between Assembly and C (and furthermore C++) to make the comparison at all. It''s more like making the comparison between C and C++, because each are alike in many ways and one just being a slight extension of the other. Only that in the Java scenario, it is running on the principle or hope that the short term investments translate into long term returns, which I think can be flawed. This is a reverse situation of what happened with the Assembly-to-C shift. People who began with Assembly were eventually forced into C and then became amazing C programmers, because they had such a fundamental understanding of computers and their different flavours. Often you will find people moaning about sloppy C/C++ programming by programmers who have no deeper knowledge of computers in general, because that is what they started on. Much worse, being a generation of students learning Java only and the SUN way of doing things and in the end I think its almost a political assertion by Sun, ultimately in their (financial) gain. In many ways, I refuse to bow down to a virtual machine that does everything for me, because I have no knowledge of what happens underneath or a decent level of freedom. I much prefer C++ and Microsoft to the extent that there is always the seperation between the language itself and the API, giving me a much greater ability to choose how I wish to do things. When you are working in Java, you just don''t have that choice. But that sounds great to a company who wants results fast, eh? And so thats why I sort of think Java is dilluting the IT world. At the end of the day, Java only has a chance in some aspects of the enterprising market, because it has no control over the general market. It''s very much what Microsoft decides to use. And its my impression that the Open Source community is a bit jaded and apprehensive of Sun as a company and their technologies. Java could just as easily be forgotten to the next best thing, when it comes in a year, 2 years, 5 years, whatever. C++ is still a safe bet, so I''m sticking with it and whatever branches out from it. If Longhorn is going to be very much a C# and .NET affair, then thats where I''m going to be. "People who say Never should get out of the way to people who can" I''ll run with what ever there is, but remember that he who laughs last laughs best. ;) |
![]() |
MamanE 06/20/04 05:18:59 PM EDT | |||
Well said :-) While this particular test doesn''t prove that a good JVM runs at least fast enough, it doesn''t mean it isn''t true. I have to admit that I doubt it''s really faster than C++ right now in all cases, but for me it''s clear that by using java (or C# if you like) you can get most jobs done faster and with fewer bugs while maintaining great performance. And who cares about whether it would have been a few percentages faster if developed in C++ when the java version was developed in a fraction of the time (thus cheaper)? Which doesn''t mean C++ is obsolete in any way, but just keep an open mind and keep educating yourself in the fast changing world that is IT. "With a few comparable adjustments I got the C++ version running more than 8 times faster." |
![]() |
larry 06/20/04 03:11:22 PM EDT | |||
Garbage collectors for C++ without corresponding VMs to run would not be very efficient. Garbage collection predates java through such languages as smalltalk and self. It was through the widespread usage of VMs via java that VM technology have gotten where it is. The original hotspot VM was intended for running self code and after they ported it to java, Sun bought the company and used the hospot VM to replace the exact VM (which was the older one). VM technology is used in products such as VMware. Microsoft is moving to a VM based archoteture with dot net, CLR, etc. Even efforts in Linux thorough the Mono project are obstensibly VM based technologies. Java is rapidly making inroads in a wide variety of areas where it was never used before. There are efforts are creating powerful video game platforms (take a look at jogl, joal, and jinput). I use java both on server side and embedded based systems. Developing in java is not a panacea for being a bad programmer. You still need to do what you doing. I remember getting into C in the early 80s. I had lots of people tell me C was crap and the only real way to do work was to do it in assembly. They would say that there was never going to be a time when C would displace assembly. That singular conversation has been going on since the beginning of computer history. Doubtless, we will have the same conversation about what ever replaces what is beyond java. As said before, you can argue that you can''t make java run fast or perform the way you want. Well, I for one can. So isn''t that really a mark of personal ability. There is an old saying, "People who say Never should get out of the way to people who can". The future is coming, ride it or be run over by it, your choice. |
![]() Jan. 26, 2018 04:30 PM EST Reads: 1,532 |
By Elizabeth White ![]() Dec. 31, 2017 12:00 PM EST Reads: 2,450 |
By Pat Romanski ![]() Dec. 30, 2017 11:00 AM EST Reads: 2,267 |
By Pat Romanski ![]() Dec. 30, 2017 08:30 AM EST Reads: 15,142 |
By Liz McMillan ![]() Dec. 29, 2017 12:00 PM EST Reads: 3,331 |
By Liz McMillan ![]() Dec. 29, 2017 08:00 AM EST Reads: 3,462 |
By Pat Romanski ![]() Dec. 28, 2017 02:00 PM EST Reads: 4,534 |
By Liz McMillan ![]() Dec. 24, 2017 01:45 PM EST Reads: 2,415 |
By Elizabeth White ![]() Dec. 23, 2017 10:00 AM EST Reads: 2,379 |
By Elizabeth White ![]() Dec. 22, 2017 11:00 AM EST Reads: 1,919 |
By Elizabeth White ![]() Dec. 18, 2017 03:45 PM EST Reads: 3,504 |
By Elizabeth White ![]() Dec. 18, 2017 01:30 PM EST Reads: 3,528 |
By Elizabeth White ![]() Dec. 18, 2017 01:00 PM EST Reads: 5,385 |
By Liz McMillan ![]() Dec. 17, 2017 04:00 PM EST Reads: 2,364 |
By Pat Romanski ![]() Dec. 17, 2017 02:00 PM EST Reads: 2,526 |
By Elizabeth White ![]() Dec. 17, 2017 10:00 AM EST Reads: 2,578 |
By Liz McMillan ![]() Dec. 15, 2017 11:00 AM EST Reads: 3,279 |
By Elizabeth White ![]() Dec. 14, 2017 04:00 PM EST Reads: 2,356 |
By Liz McMillan ![]() Dec. 14, 2017 11:45 AM EST Reads: 2,423 |
By Elizabeth White ![]() Dec. 14, 2017 11:00 AM EST Reads: 2,439 |