Exceptions make your code faster in the non-exceptional case.
See all those if-branches for error checks that don't exist in your code? That branch-free code is going to run faster than your branchy code. And the code's shorter, because it doesn't have the checks and handling intermixed, which makes it fit better into cache.
This isn't rocket science, I'm amazed that this isn't obvious.
This is a good example of something that sounds reasonable, even intuitive, but in practice just isn't necessarily true.
Ever since the 486 we've had zero-cost instructions, for example.
More pertinent to this, modern CPUs have multi-stage pipelines and branch prediction.
Third, just the possibility of throwing an exception has a cost as the program has to have the code to unwind the stack.
Last, humans have become increasingly bad at second-guessing compiler optimizations.
> This isn't rocket science, I'm amazed that this isn't obvious.
Thing is, it kind of is rocket science. And it's not obvious. That's why you have to test.
Now you could take the position that exception lead to better designed code and might have other such benefits. Personally, I've become more convinced over the years that exceptions are a false economy.
> More pertinent to this, modern CPUs have multi-stage pipelines and branch prediction.
A correctly predicted branch is still slower than no branch. An incorrectly predicted branch is one of the slowest things you can have happen on a modern CPU.
> Third, just the possibility of throwing an exception has a cost as the program has to have the code to unwind the stack.
Incorrect. The exception handling code does not execute in the normal path.
-fno-exceptions ~never makes your code faster. It does, however, shrink your binary size. Which is a non-trivial cost, but not the focus of the discussion.
> An incorrectly predicted branch is one of the slowest things you can have happen on a modern CPU
Per Agner Fog, misprediction on Ryzen will cost you 18 cycles. On Haswell family, 15 to 20. Floating point division can cost you more cycles, both in latency and throughput. Even just plain old L3 latency can be worse than misprediction.
That’s oversimplified. If the falsely predicted branch issues a load instruction for example, you might use up valuable memory bandwidth, trash cache lines, etc. There are way more side effects that can make false branch predictions more expensive. It’s also extremely microarchitecture dependent, proprietary and undocumented what exactly happens, which is why we need to benchmark the actual code under consideration.
Yes. Also a misprediction affects all instructions in flight after the branch, a floating point operation or slow load only affects instructions that depend on it.
> Floating point division can cost you more cycles, both in latency and throughput
No, on Ryzen, 32-bit float division, divps instruction, has latency 10 cycles, throughput 3 cycles. 64-bit float division, divpd instruction, has latency up to 13, throughput up to 5. Both faster than a cache miss, in latency and especially throughput.
> Even just plain old L3 latency can be worse than misprediction.
Yes. BTW, the slowest thing that can happen on modern CPU is not L3 access, it’s access to a cache line modified by another CPU core. Even slower than main memory latency.
> Incorrect. The exception handling code does not execute in the normal path.
This is true only if only exceptional things throw exceptions. In some APIs normal, but not successful things throw exceptions. Things like file not found are often pretty normal, but may throw an exception.
I often hear this argument, but I find it misleading. First, is suggests that we need to have two systems: for "normal" errors and for exceptional ones. Second, the programmer now has to classify the conditions into normal and exceptional without any real criteria to do that. Is out of memory exceptional? For many applications it is, but not for Phothoshop where it's a normal state and you just have to free some memory and try again.
Exceptions are exactly same as errors, they're just a different way to handle them. Errors are natural to programming, because nearly every function may end up in two states: it successfully carried out the task (no error) or it failed to carry it out and here's the reason why (error).
The reason we have exceptions instead of errors is generally because we want the code to look syntactically sweet. A mere "a.b + c.d" in modern language may involve three method calls: to get "a.b", to get "c.d", and to add them using some custom definition of "+". Each of these methods may run into an error and exceptions provide a syntactically sweet way to handle this.
Not two systems, two APIs; like Int32.Parse(String): Int32 and Int32.TryParse(String, out Int32): Boolean in .NET.
It's quite rare to want to handle an error condition half-way up the stack and do anything more than log / cleanup and resume unwinding (whether implicitly via exceptions or monadic errors or automatically via exception unwinding library).
You normally handle errors like file not found, couldn't connect, etc. near the leaf call which failed, if you want to handle them at all. So it makes sense to decide what error condition you want; do you abort the whole task (exception works great!), or do you do something interesting like retries or alternatives (exceptions not so hot, booleans or some kind of error code better reflects that the handler code is conditional rather than exceptional).
Yes, I see the reasoning now. But isn't this pure syntactic sugar? I.e. if exceptions weren't costly, a good way to implement TryParse() would be to simply call Parse(), catch the exception and dress it up as a boolean.
If exceptions were free then yes, but there's not really any language/platform where that's the case. It'd instead be the reverse - Parse() calls TryParse() and throws an exception on false.
Not sure about the current state but not too long ago the unwind information wasn't stored separately from the regular path which meant that they used up valuable instruction cache.
At least on GCC since 3.x, unwind info has been on a separate section. It is not even machine code (dwarf unwind opcodes are run by an interpreter), so it is unlikely to be in the instruction cache.
The unwind info might end up calling destructors or exception handlers, which might or might not be on a separate page. You are probably referring to that.
>> Third, just the possibility of throwing an exception has a cost as the program has to have the code to unwind the stack.
> Incorrect. The exception handling code does not execute in the normal path.
But it does need to undo all allocations and such that were made between the exception and the nearest exception handler, doesn't it? Isn't this "unwinding the stack"?
Simply returning from a function is also "unwinding the stack" but nobody writes blog posts & articles about avoiding return ;)
But what you're referring to is a cost, yes, but only a cost when an exception happens which should be rare. And it's not that much more expensive of an unwind than what would happen if you `if (err = thing()) { return err; }` everywhere in your call stack anyway.
> But it does need to undo all allocations and such that were made between the exception and the nearest exception handler, doesn't it? Isn't this "unwinding the stack"?
it should do so if you use error codes too - else you've got an incorrect program which leaks things like memory (not too bad on our current computers), file handles (really sucks on windows), sockets (really sucks on linux).
I agree, I was just failing to grasp how the execution path of exception handling code is not "unwinding the stack". The compiler will have to prepare each stack frame and mark what needs to be deallocated and the exception handling code won't just jump straight to the nearest handler, but will visit each stack frame and undo the allocations, calling destructors and such.
On the other hand unwinding the stack is not complex and won't add much to program size.
If they're rare enough that they are ~never mispredicted then why does the cost of exceptions matter when they do happen? If they are common, then they're going to be mispredicted.
The demo benchmark, for example, would produce basically always missed branches. Well, if would if the compiler didn't just strip the empty branch, invalidating the entire benchmark.
Because the branch prediction cache puns an error check with a loop predicate and some other random branch.
Branch predictors steadily get better, sure, but they're not infallible. Remember that reducing the total number of branches in a program helps the branch predictor predict better.
I too am amazed. He wrote a code which will throw in 50% cases or already very fast code `(random() % 2 == 0)`. Hell yes, a huge performance penalty is expected. But what about `nullptr == fopen(..)`? I bet the performance penalty would be negligible here.
Check out the godbolt link I posted under desc's thread. The answer isn't that obvious. Both approaches have a conditional. The compiler/linker marks the exception handling code as cold. In theory this could go in a section of the exe that the loader never pages into RAM. (I'm getting out of my depth here).
You already take a page fault with DWARF exceptions, since the unwind / LSDA info is probably on a different page from any of your code, as well as the personality routine itself.
that potentially happen even with normal if based error handing. If the compiler thinks the error handling path is unlikely to be run, it can move it to a cold page.
This is an over-simplification. The "fitting in cache" is quite complicated, and as a good rule of thumb exceptions make cache locality less optimal even if you rarely call them. There is little that is "branch-free" about exception code, it literally implies branching.
For performance-sensitive code, I've yet to find a real-world case where exceptions out-perform exception-free code. Nor have I seen much in the way of contrary evidence among people that care about performance.
What's the complication on the fitting in cache? The error handling is taken out of the code that's loaded in icache. When you throw, a runtime library call is made that walks your stack. Until throw, that code has no reason to be in your icache.
Yes! Exceptions make your code faster and smaller. And by isolating exception handling to many fewer cases, it's more likely your exception handling will be correct.
Go has GC, which has real costs always lied about because they are hard to tie to specific instruction sequences. So, Go will always be slower in certain key applications where latency matters, but in most other cases not so anybody notices.
Otherwise, there are too many differences in detail to generalize. Rust can take advantage of reduced aliasing that C++ can't, but the compiler doesn't because that code is still too buggy to turn on.
Last time I compared carefully, Rust and C++ built with Clang had identical performance, to well under 1%: much smaller than, e.g., the difference between Clang and Gcc.
Branchy code where the branches are statically known to be very unlikely ought to be practically as fast as the branch-free equivalent. You're probably forgoing a few optimization opportunities since you do have to build a proper exception record (and perhaps a stack trace) if the code errors out but other than that, it just doesn't matter that much.
Branchy checks for error codes -- the alternative to using exceptions -- are not statically predictable. Each such branch is dynamically predictable, but burns one of a strictly limited number of available branch prediction slots.
So, while the successfully predicted branches are pretty cheap, many fewer can be predicted. Synthetic benchmarks generally have many fewer branches to predict than live code has.
It is absolutely done, it's just implicit rather than explicit. The simplest heuristic is to predict backwards branches as taken (loops) and forward branches as not taken, and it works quite well (though I think it might be more sophisticated than that).
Of course the error-code checking branches will most usually be forward branches, thus not predicted correctly the first time through,
or after spilling.
I can't say I'm a fan of these sorts of articles. "Which strategy is most performant?!" is rarely the appropriate measure for an error handling strategy, it's more nuanced than that. Concerns like what makes a recoverable error vs an unrecoverable one are more important. Ensuring correctness is more important. The usability of an API is more important. Readability and maintainability are more important. Knowing what errors mean in terms of your domain model is more important.
Use what fits for your situation, don't take the "when all I have's a hammer..." attitude to anything. Sometimes return codes make sense (eg in hot-loop code). Sometimes exceptions make sense (in constructors, for example. And no, two-step initialisation is not a good alternative.). Sometimes neither make sense and you actually should just terminate the whole program (out-of-memory). Sometimes contracts make more sense (precondition checks, validating function arguments).
Anyway, regardless of your strategy if you can provide some method of querying if a thing can succeed to your API, do so; it's always better to handle errors by just making sure you don't cause them!
I would add that a error-handling strategy should never be a concern wrt performance because obviously the purpose of this sort of code is to serve as guard rails and ideally never be called at all ever.
If your error-handling code is triggered so often that you feel it might be causing a performance hit, you have far more serious problems than the performance overhead of how you handle errors.
I mean, it depends, CGAL for example, which I'd consider a high quality software library, uses exceptions for their exact number type to redo the calculation using more precision when needed https://www.cgal.org/FAQ.html#uncertain_exception
You're invoking the appeal to authority fallacy. Just because a popular package adopted a practice that does not make it right or good or acceptable. If that line of argument made any sense, you could argue that bugs present in CGAL represent best practices.
Rule of thumb: if a code path is not exceptional then it should not be implemented with exceptions. The happy path is not an exception.
My web server gets about 1000 to 1 hits of hacking attempts to valid requests (malformed headers, etc.) so I hope nginx and fcgi handle errors efficiently, because I pay for those cycles.
In those cases, a malformed request is common enough that discriminating between well-formed and malformed should be thought of like a first-class part of handling a request, not an error-handling situation. Errors are when things go wrong that you don't expect to usually go wrong*
*for varying values of usually, YMMV, use your own judgement, etc
You're not describing exceptional events. You're describing pretty much the happy path of any reverse proxy. In fact, in some applications these sort of events outnumber HTTP Status 200-worthy requests.
That assertion is obviously wrong. Think about it: you have a web server listening to the world. Do you really expect all requests to be neatly pointing to sanitized URLs? Of course not. Why on Earth should you assume that handling invalid requests is not one of the primary use cases?
When I want a parameter is an integer, I don't check all the ways in which it's NOT an integer; the happy path is when it _is_ an int and the program can continue, and anything else is an _Error_. If it overflows, that's an error too.
i.e., happy means it's basically the longest path (aside from diagnosing the error).
The question is: should the subroutine set an error number or throw an exception? Knowing the cost for each might help me decide for various situations.
It should be well-known that actually throwing an exception in C++ is extremely costly. I think it is also fairly widespread knowledge that even allowing the compiler to support exceptions has runtime costs. Exceptions do not conform to C++'s "zero-overhead" principle. That's why numerous large organizations from NASA to the Dept. of Defense to Google decree that C++ exceptions must not be used.
> Exceptions do not conform to C++'s "zero-overhead" principle. That's why numerous large organizations from NASA to the Dept. of Defense to Google decree that C++ exceptions must not be used.
I don't know about NASA's case, but Google's style guide forbids exceptions only for legacy reasons. In fact it praises them:
"On their face, the benefits of using exceptions outweigh the costs, especially in new projects. However, for existing code, the introduction of exceptions has implications on all dependent code. ... Because most existing C++ code at Google is not prepared to deal with exceptions, it is comparatively difficult to adopt new code that generates exceptions."
Since I'm not working on Google's old code base I do use exceptions.
> It should be well-known that actually throwing an exception in C++ is extremely costly.
Unfortunately it is well known, but without really understanding what "extremely costly" means. The article shows it is expensive compared to returning from a function that does almost nothing. But what about compared to allocating some memory? Or locking a mutex? Just recently here on Hacker News I recently saw someone worryung about using an exception to close a file after doing a bunch of reads because they thought it would be too slow – but, in reality, the time taken to read a whole file is astronomical compared to throwing an exception.
> C++'s "zero-overhead" principle
Now it's possible you're right here, but just to be clear: this principle just means features don't have a cost if you don't use them. For example, if you don't need a function to be virtual then you don't have to mark it as such, so the fact that C++ supports virtual methods didn't cost you anything. But if you do choose to mark a method as virtual then there will be costs involved (memory will be taken up with the vtable and pointers to it, invocations through a pointer will take longer). Similarly, the fact that throwing exceptions isn't zero cost does not violate this principle.
The reason I say you could still be right is that there is still a memory cost to including tables of unwinding information for exceptions even if you never throw them. Perhaps that's what you meant, but the surrounding text seems to be hinting at the cost of throwing them.
I think qualification of statements is in order. I have heard the definition you gave for ‘zero-overhead’ abstractions in several places. Basically, it is often described as features which impose no cost when they are not used. However, Stroustrup uses and advocates another definition, which he discusses in his response to Sutter’s exception proposal linked by a sibling comment. When Bjarne says ‘zero-cost’ he means that some feature incurs no overhead in comparison to equivalent functionality which is hand coded. These two definitions are not equal, Bjarne’s definition (which as the originator of the term [or so I’ve heard], I tend to use as the correct case) is only concerned with asserting the overhead of language constructs compared to specific implementations of the same functionality. There is no requirement that those abstractions and features have no impact on the produced machine code or runtime requirements of not used. I wish zero-overhead meant what you say it means, but it clearly doesn’t.
Interesting, thanks. I thought I got my definition from Design and Evolution of C++ by Stroustrup but it's a very long time since I read it. (It's also possible that Stroustrup has used that term in different ways over the years, but it's much more likely that I'm just mistaken.)
I think your memory is correct and that Bjarne has probably been loose with the term, I’ll check just to satisfy my curiosity later, but I do know he has been using the version I mentioned several times in the last few years.
Some abstractions have no overhead under any circumstances, others have no runtime cost if not used. It is not inconsistent to include the former, trivially.
But my point in making the distinction is that Stroustrup does not intend to say that 'zero-overhead' is equal to 'no runtime cost if not used'. Stroustrup favors the following:
'zero-overhead' for some feature is equal to 'no runtime overhead above what one would incur from coding the feature yourself.
That definition explicitly does not include the concept that 'zero-overhead' features carry no cost when not used. In fact, the entire point of Bjarne discussing what "zero-cost" means in his response to Sutter's exception proposal, is to state that for C++ (which originated the term) "zero-cost"/zero-overhead does not mean that language features do not incur cost by inclusion in the language, like with exceptions. There is a cost to code during runtime if one does not explicitly remove exceptions from the language at compile time. Stroustrup's point is that this built in runtime overhead from having an exception mechanism built into C++ does not violate the language's 'zero-cost' guarantee.
But in that case the equivalent would be manually writing out return codes (or return objects like Expected) and checking errors everywhere. Which at the moment is quite a lot faster than using exceptions. It may not be faster to write but it's quite a lot faster to run, to compile, and uses less disk space/memory for the resulting executable.
Ideally I would like to use exceptions as well, since they make the code look a lot neater and clearer, but I cannot justify their use in all places because of these non-zero costs.
The time to compile exceptions is trivial. Templates and overload sets cost.
The time to run exceptions is a lot less than to run failure checks, except in extreme cases as in the synthetic (not to say phony) benchmark presented. If the time spent throwing matters, you're Doing It Wrong.
The branch prediction slots that failure checks burn are rare and precious: much bigger than a general purpose register, and much faster than L1 cache. Blowing your branch prediction cache has costs hard to measure, although the perf registers help some.
The only substantial cost is object-code file size, and that mainly because precious little effort has been spent to minimize it, because really hardly anybody (i.e., only embedded) cares.
The fact that just enabling exceptions adds a non-zero cost to the runtime of your program, even if the code itself does not use them, means that they are slower than just checking the return values. I've seen values from 5% to 15% slower runtime performance for just enabling exceptions and not even using them. I don't think removing the error checks will give you back those percentages.
The problem with proving something like this is you have to build and maintain two separate sets of code for the same project, one that uses exceptions in the correct manner, and another which does error checking to the same degree. Nobody has time to do that for a significant project.
> an uncontended mutex lock takes around 10-30ns depending on the machine.
I am grateful for this. But I do wish the article had a few more comparisons like this, up to and including things much more expensive than exceptions, so readers could make their own decisions rather than just read about how terrible exceptions are.
As another data point, I just read that a Linux system call that does no actual work takes about 250ns, so about a fifth of the time for an exception from the article. Of course that's a totally unknown machine that could be much slower or faster than the one used for the article's benchmark. But I still found it interesting that they're the same order of magnitude – I hadn't expected exceptions to be quite that slow.
Stroustrup constructs a bit of a false dichotomy there. He presents the choice between exceptions and explicit error propagation and checking, but there is a third way. If the allocator is simply infallible and std::bad_alloc just doesn't exist, and the program terminates on allocation failure instead, then you need neither exceptions nor error codes. Since std::bad_alloc is such a prominent root cause of all exceptions, and since it's preposterous to believe a program can continue after the global allocator stops working, this turns out to be a dramatic simplification of the overall program. Once you get rid of allocator exceptions it's not a big mental leap to just decide to get rid of all of them.
But std::bad_alloc does exist and it's possible and sometimes necessary to write correct software that's able to handle allocation errors. What we probably need is some sort of a compiler switch, but it will significantly complicate writing generic multi-purpose libraries
bad_alloc doesn't just naturally spring into being. If you override operator new, your implementation may not use it (for example tcmalloc does not throw if it can't allocate, it logs a message and terminates the program). Many systems of the type you mention, embedded control systems and the like where correctness actually matters, don't use new and delete anyway. They often use placement new with space statically allocated at program startup. These are the same kinds of projects that often decide to not use exceptions.
When the global allocator fails, you might not be able to do much... except set a flag, and free a big block of memory that was reserved against just such an event.
Then the program can do whatever cleanup is warranted, maybe freeing up a variety of other resources, and then maybe re-reserve that block, and forge ahead.
But when you actually care about performance, you often are not using the global allocator. A local allocation failure is nothing to panic over. Generally, only the higher level code -- where the handler is -- knows, or needs to care, what kind of allocator failed. The low-level code gets to just bail out, as it should.
Contended mutex locks, of course, may take far, far longer than 30ns, and 30 nanoseconds is an eternity, enough time to run 500 instructions on a modern CPU.
Herb Sutter is, reliably, always right on his facts, but always mistaken on his opinions.
Its unrolling the stack, whatever could go wrong?.. Its only expensive when you throw an exception.
They should only be used in exceptional circumstances, but they appear to be littered among libraries for everything from telling the caller that the end of file has occurred to a connection timing out.
The cost comes from having to track what automatic and temporary objects have been created and attaching them to the exception context so you can destroy them if an exception is thrown. Another cost is having to track which base classes of a derived class have already been constructed when the derived constructor throws. You get noticeably better generated code when the compiler can skip all that junk.
> The cost comes from having to track what automatic and temporary objects have been created and attaching them to the exception context so you can destroy them if an exception is thrown.
Doesn't it have to do that anyway, in case you return from a function, or break from inside a loop? Anytime you can leave a scope, it has to be able to do that.
> Another cost is having to track which base classes of a derived class have already been constructed when the derived constructor throws.
All of them have been constructed before you begin the derived constructor.
Thanks for holding the discussion to exact standards. What I meant about the base class constructors is the compiler must generate a derived-class constructor that can unwind the completely constructed base in case the derived class throws, not to mention the ability to destroy constructed subobjects with non-trivial destructors in case one of _their_ constructors throws. When your project decides to outlaw exceptions then none of these events are even possible and the compiler will generate a much more compact and infallible constructor.
As for leaving the scope, you're right again and I was unclear. The problem is you are destroying automatic objects while returning from a scope and one of them throws. Now you have extra code and extra branches for all of that. The extra ways of exiting a scope generally bloat up the program, and the size and complexity of a function can prevent it from being inlined or otherwise optimized by the compiler. These are systemic performance concerns that won't just show up as hot spots on a profile.
Ah, yes, I wasn't thinking about throws during destructors. IIRC, that's considered bad form but not forbidden, and so a compiler would have to handle that. (Feel free to correct if I have that wrong.)
As to your first paragraph: That's just calling the base class's destructor if the derived class's constructor throws. That's easy - that's what the base class's destructor is for. It's essentially one function call. The harder part (or so it seems to me) is correctly destroying the only-partially-constructed derived class.
Throwing during destructors will generally result in a std::terminate, since you have good odds of throwing during destruction of an object that is being cleaned up as a result of an existing in progress exception.
So it is not forbidden but odds are you'll have your program terminate.
while there is extra code, there should be no extra branches in the successful case. The exceptional code path is reached during unwind by comparing the IP of the throwing call with a per stack frame table (recursively walking up the stack). Also the extra code will be extremely cold, at the limit not even paged in from disk (which of course might make the first time an exception is thrown an explicit path extremely costly)
The compiler has to generate code that works when exceptions are thrown. This limits some optimizations and adds overhead even if nothing happens at runtime.
I don't recall NASA outright banning C++ exceptions - especially not organization-wide. Code running on space hardware has very specific risk profile, and AFAIK NASA dislikes exceptions there because they add risk to code (without a mechanism like Java's checked exceptions it's too easy to add a code path that can generate an unhandled exception, terminating the program).
Yeah, but the slides don't give a reason why; the only hint is a prior slide mentioning the "historic arguments against using C++ in flight", namely "Exceptions: too much uncertainty, difficult to validate all possible control paths". And that has nothing to do with performance.
ZeroMQ's author has written a wonderful article about how he should've picked C in place of C++ in a mission-critical setting, even despite personally preferring C++ himself.
This is an interesting post, definitely! However, as it was posted in 2012 I feel like some of his concerns have been addressed in newer versions of C++.
Instead of having a Constructor + init method you could have a static create function which returns an optional or a pair.
I have a hard time seeing in what case you would want to throw in a destructor though? Maybe I'm just writing a different kind of code :)
> I have a hard time seeing in what case you would want to throw in a destructor though? Maybe I'm just writing a different kind of code :)
I can't imagine that case either - but what I can imagine is that your destructor calls a function that calls a function ... that calls a function you didn't realize can throw.
Well, mission critical software has to meet latency requirements, but it has to do so in a way that's provably bounded. In some ways, it's actually worth being a little slower if it's easier to prove you meet latency requirements.
Point being it's not entirely orthogonal to "performance".
C++ had runtime checked exceptions, which aside for the except/noexcept case, were pretty useless (and never implemented by MSVC).
I'm a huge fan of checked exception, given a type system powerful enough to parametrize over them (C++ should be more than capable), so I'm looking forward to the development. Stroustrup is not a fan though and he might kill the effort.
oh, yes. As far as I can tell, these static exceptions are basically syntactic sugar over an either<T, Except> return type, possibly with a custom ABI that stores the discriminant implicitly in a flag register (so on the call side you do not even need to load the discriminant and compare it and instead directly do a JMPcc ).
The standard doesn't give advice about how to break features of the standard, so disabling exceptions is a non-standard feature of the implementation. If you're planning to disable exceptions you'd better read the docs that come with your compiler.
It's even so well known that this infects thinking about exceptions in other languages. However, there are languages where throwing of an exception is as cheap as any other function call (fe when they are compiled via the so called "double barrelled continuation style" strategy)
I believe the exception ban at Google these days is justified more on the grounds that integrating with existing code would be problematic, rather than on performance grounds. I seem to remember reading something to that effect.
That may be the explanation they've settled on but there are build artifacts at that company which would be impossible to link without -fno-exceptions, so there are also practical problems.
This is correct. Google have many, many million lines of casually exception-unsafe code they cannot afford to fix, so any rationalization is only that. The choice is off the table.
sigh If your application is slow, odds are it's not because you used exceptions.
If you're throwing enough exceptions for this to matter it'll show up on a profiler, and then you can change that specific chunk of code to avoid treating that particular case as 'exceptional'.
Tim Sweeney, creator of Unreal Engine, Epic Megagames, etc, recently had a code base where exceptions were causing problems even when not actively being thrown:
>They weren’t throwing and a disassembly showed no visible artifacts of exceptions. But turning off the possibility of throwing exceptions gained 15% and just made the assembly code tighter with no clear pattern to the improvements.
I am very familiar with this effect. We could of course use effects like this to cast doubt on any benchmark someone has ever run, unless they specifically mention that they tested for this, and 100 other bench marking gotchas. We can, if we like, assert that all things are unknowable, while at the same time asserting that the thing we want to be true is for sure true.
However, it appears to be a rather commonplace occurrence that having exceptions on, even if you aren't using them, can cause performance problems, it isn't just a one of in Tim's case. Also Tim has quite a bit of experience working on bleeding edge C and C++ performance code so there is a good chance he did account for this. You can ask him.
How common is it, actually? And on what platforms/architectures? It was a common problem back in the day when most code was compiled for x86, since exceptions weren't designed to be zero-cost in that ABI.
For what it's worth, the article itself has this bit:
"Thanks to the zero-cost exception model used in most C++ implementations (see section 5.4 of TR18015), the code in a try block runs without any overhead."
It has been at least a 10% effect in the last two things I profiled, which were a simple software rasteriser and a distributed key value store. The other significant benchmarking gotchas I see are: 1) the CPU being in a sleep mode when the test starts and takes a while to get running at full speed, and 2) other stuff running on the machine causing interference. But these two are easy to work around compared to the alignment-sensitivity problem.
I didn't mean to disagree with the conclusion. My point was more that it is hard to be confident in the causes of results like this. It'd be great if we had tools that could randomise alignments etc, so we could take lots of samples and get more confidence. As far as I know those tools don't exist and we just have to use experience.
This sounds like a straight up optimizer bug, and should be reported as such.
And if profiling does show up a case like that, the appropriate response to that is slapping noexcept on the function, not disabling exceptions globally.
Indeed, and this was measured, and the improvement was worth it. Profiling! :P The cost isn't always where we think it is.
Still, 'turn off exceptions because exceptions are slow' is a daft rule of thumb for the majority of software, where the slowness probably has more to do with choice of data structures, etc than compiler/platform implementation of language features.
I think that in theory there should be no difference between code that can throw and that can't from the point of view of the optimizer. In practice I think that sometimes some code motion passes are disabled in the presence of abnormal control flow because the compensation code that would be otherwise required becomes unwieldy and hard to prove correct.
Some case might be “exceptional” in the sense that it doesn’t happen most of the time (and thus doesn’t show up on regular profile checks), but when it does happen the failures are highly correlated and suddenly you’re throwing thousands of exceptions per second all over the place.
This is also the time one finds out that the exception handling code on typical C++ runtimes take out a global lock and the multithreaded application grinds to a halt—the raw CPU cost of an exception is not the most pressing problem at this point.
malloc takes out a global lock and performance doesn't 'grind to a halt' from a single allocation. Hundreds of thousands of allocations per second per core and concurrency will suffer, but most programs either don't actually have nearly enough concurrency or minimize their allocations to the point where it isn't a primary bottleneck.
I think the context you’re working in matters heavily with such an assertion. In fact, it is quite common in game development to bulk allocate and then use custom allocators within these regions specifically avoid the drag on performance and determinism inflicted by malloc.
Bulk allocations with custom allocators would either be exactly what I said (low malloc calls) or would just not use malloc at all and use the system memory mapping functions.
The larger point was that an exception being thrown and taking a global lock is not going to tank performance. That would imply that all other threads are trying to take the same lock at the same time and that the thread that gets it holds it for a long time.
Even in the case of malloc, where there could be actual heavy lock contention, this is not always a bottleneck.
Of course, it would depend on the workload. An exception here and there wouldn’t matter, but for a somewhat contrived example, using exceptions for something like a bad connection to a database on a server with dozens of threads could easily turn into a concurrency and a resource exhaustion problem (where every thread starts seeing the same error at the same time).
All modern common implementations (I think). MSVC's malloc definitely locks. I think gcc, clang and icc do too. This is a big reason why tcmalloc and jemalloc were created.
I don't know about MSVC, but glibc malloc has per thread caches. At some point you need to hit the global allocator or sbrk or mmap of course and that might take a global lock.
Looking at the disassembly the machine code is ~2x the size for the exception versions, but most of it is on the cold path.
The exception version has a conditional branch to do the "== errorInt" part. The non-exception version manages to avoid the conditional branch by using a conditional move, which would avoid a pipeline stall on a branch mis-prediction.
Edit: I think this disproves desc's point ("If your application is slow <because of execptions> it'll show up on a profiler"). ie there's probably a small cost to exceptions even when they are not taken and it will be spread across your entire program and will not show as a single spike in a profiler.
> The non-exception version manages to avoid the conditional branch by using a conditional move, which would avoid a pipeline stall on a branch mis-prediction.
branches are usually superior to conditional moves for predictable conditions as they break dependency chains. In case the exceptional code path is taken, the cost of the misprediciton is dwarfed by the cost of unwinding the stack.
This is interesting actually, the fact that the compiler uses a conditional move in the error checking case could mean that the compiler has no useful branch probability model for that branch in the error checking case, but even when using __builtin_expect, the compiler still prefers the conditional move.
Agner Fog is the usual go-to reference. For this specific case, you can also google any of Linus rants on conditional moves (they used to be very high latency, although today they are not so much of an issue). This one for example: https://yarchive.net/comp/linux/cmov.html
It is complicated to describe when cmov is slow and when it is fast. As a rule of thumb, if the next loop iteration data operations depend on a cmov in this one, and around, cmov will be slow. If not, it is very, very fast. Use of cmov can make quicksort 2x as fast.
Gcc absolutely won't generate two cmov instructions in a basic block. Clang, for its part, abandons practically all optimization of loops that could conceivably generate a throw.
The problem with benchmarks is that I never see any that estimate the impact of the extra code size on programs the size of, say, Photoshop. It takes annoying long to load such a program. Is code size part of that problem? Probably. Is the bloat added by exceptions significant? I'd like to know.
When it takes a program too long to load, it is because the program is doing too much non-exception work. The exception-handling code is not even being loaded unless it's throwing while it loads, which would just be bad design.
Interesting, GCC 7.x seems to simply puts the cold branch on a separate nop-padded cacheline.
GCC 9 [1] instead moves the exception throwing branch into a cold clone of exitWithMessageException function. The behaviour seems to have changed on starting from GCC 8.x.
Ooo, fancy. There is still a long way from just that to actually getting the savings in a real program running on a real operating system. For example, if I have thousands of source files, each with a few hundred bytes of cold exception handlers, do they get coalesced into a single block for the whole program?
Code paths introduced in order to execute any potential stack unrolling are inefficient and they make your code slow. Especially tight loops. This was common knowledge back in 2000s.
Common knowledge, but not correct. Code to destroy objects has to be generated for regular function returns, and is jumped into by the exception handler too. Managing resources by hand, instead, would also require code, but you have to write it. Its expense arises from its fragility.
This one adds functions that call the exception-based and error code based functions in a simple for loop. Both handle the error.
Unless I've screwed up somewhere, I think the result is that in the exception case, the body of the inner loop contains 13 instructions, while the error code case contains 5.
Also, the generated code for the exception case is harder to read and understand. When writing performance critical code I like to eye-ball the disassembly just to make sure the compiler didn't do anything unexpected. This task is hard enough already in non-trivial functions, I certainly don't want it getting any harder.
Also generally having slow code is less a problem then expensive code - exceptions can allow you to use more succinct expression that will lower long term maintenance costs - the full lack of exceptions is one of the things that I think continues to impair Go which has a lot of other neat ideas (and a really good way to deal with sub-addressing in the form of slices).
Panic/recover are much more limited and while you can build an exception system from them (much like all turing complete things can be all other turing complete things) it is a pain and extremely inefficient. So panics do exist and are used for I/O errors sometimes but it's quite inconsistent.
Exceptions are not meant to be fast. In fact, exceptions are meant to be exceptional. Hence the name.
Exceptions are meant to be fast if and only if you do not throw any. And even this is only truly true on Unix.
The original implementation of exceptions on Windows put extra unrolling info on the stack and then cleaned it up at the end of the function, adding a runtime cost even when no exceptions were thrown.
The Unix way of exception handling does not put anything extra on the stack, so exceptions are free if you don't throw any. The compiler puts some metadata into your binary and generates cleanup code. If an exception is thrown, the run-time (libstdc++ for gcc on Linux) will grab the return address from the stack and then do a binary search through the metadata to find the right unwinding code generated by the compiler. This is obviously a comparatively enormous overhead. In particular because the metadata are separated from the code, so the OS never has to read it from disk unless you actually throw an exception.
The reason for that is that you are not supposed to use exceptions for signaling. They are for error handling. Exceptions are for occasions like "you asked for memory but there was none left" or "you asked for the 15th element in this vector but it only has 10". They are not meant for non-exceptional errors like "you asked for data from the network but there is no data yet" or "you wanted me to check if the file exists and it does not".
The thinking behind all this is that the error handling path is not time critical. If it happens, something exceptional has happened, and the OS will probably have to swap in the error handling code from disk to begin with, which is horrendously slow. In fact, this is not just a possibility. The feature was designed specifically so that the linker would move the exception handling code to a different page and the OS would only read that page from disk if an exception actually occurs. This is not an accident. It is deliberate planning and took engineering effort to achieve.
Running a benchmark of throw vs return is an attempt to answer the wrong question.
Important side note: operator[] on a vector does not do range checking. at() does. That's why you should always use at() instead of operator[].
You seem to be arguing that, because exception tables end up on separate pages, we should consider exceptions as being as expensive as disk accesses. (Apologies if I've just built a straw man, that's how I read it.) I don't buy that. If you use exceptions more than totally exceptionally, according to your definition, then those pages will be loaded to memory and kept there – while the optimisation sounds very clever and sensible, I have a hard time imagining that exception metadata is huge in comparison with the rest of the program code.
Once that data is in memory, you're back to the numbers in the article, where throwing an exception can take barely more than a microsecond. That's huge compared to a function call, but is fast compared to a lot of things that an overly cautious programmers wouldn't dare follow up with an exception. To steal your own example, checking whether a file exists certainly falls into this category – that's likely to take milliseconds.
(Even your other example – no data available to read (I'm imagining a select / poll situation) – would probably be fine with exceptions, though I agree it's not the right tool for that situation. Just making a call to the OS is likely to cost a comparible amount to throwing an exception, even though you're describing a non blocking call. And by definition this won't end to being called in a critical inner loop - if data becomes available then the exception will do being thrown.)
He makes the point that exceptions should be truly exceptional conditions for your program that would prevent it from running, generally due to system malfunction. Some people use it for basic business logic and it was never meant for that. It is meant to help you gracefully abort things when everything is about to go to hell.
This is not an accurate summary of what C++ exceptions where "meant for". Here's Bjarne in 1989:
"The author of a library can detect errors, but does not in general have any idea what to do about them. The user of a library may know how to cope with such errors, but cannot detect them – or else they would have been handled in the user’s code and not left for the library to find.
The primary purpose of the exception handling mechanism described here is to cope with this problem for C++ programs; other uses of what has been called exception handling in the literature are considered secondary. See the references for discussions of exception handling techniques and mechanisms."
I think it depends what you consider an error. For example let's say I have a library that opens sockets to hosts. If I can't reach the host that isn't an error and I report back a failure of that function to succeed. If I can't allocate the memory for a socket then instead of reporting a failure to connect I throw an exception for the true error.
> He makes the point that exceptions should be truly exceptional conditions
The comment did say that, but the only explanation of why exceptions should only be used that way seemed to be performance based. If that is not the issue, then what is? Saying "that's what they're for" without any other detail is a circular argument.
I explained how exceptions are implemented, and the actionable intelligence you can derive from that understanding (if perf matters to your code and/or scenario, then use exceptions only for hard errors, not for generic signaling).
If perf does not matter in your scenario, than obviously the guidance does not apply to you. To me that does not mean it's a waste of time to explain how exceptions work and what that means to performance critical code.
It is a circular argument. I for one I would use exceptions significantly more often if throwing and catching them were to be cheaper (also syntactically). As it is you often end up with both a throwing and optional-returning non-throwing variants of many functions.
Well it depends on your circumstances. If you care so much about performance in the code path that you do benchmarks to find out how you should be doing your error handling, then you should not be using exceptions.
Even if the metadata is loaded after the first time, the error handling code will probably still need to be fetched from disk.
Most programs are not performance critical and you can use whatever you want for exception handling. Heck, modern microservice based architectures use message busses and network sockets for error handling. I have seen code signal error conditions by writing things to a database.
By the way: Syscalls have become amazingly fast. Some syscalls like gettimeofday are now even done without switching to kernel mode at all. And the raw kernel mode switch is now in the order of hundreds of cycles. I read something like 300 cycles for a modern architecture.
Also note that while regular performance is probably not so important, in some areas like hard real-time requirements for automotive or aviation it might still be relevant that your code has a predictable maximum latency.
Converting an exception into a return. Which loses the "have to deal with it; cannot ignore it" property that made the exception valuable.
"The key takeaway here is that an exception cannot be ignored or forgotten; if we have an exception, it must be dealt with."
If converted to simple "C style" status returns -- it is only 5 times slower (11.5ns vs 57.5ns) -- and can be ignored and forgotten.
In the associated slide deck, boolean valid() must be used before value. In this case, this is just as unsafe as C return codes. A C++ exception is always considered an lvalue, meaning that union is initialized, but I am not sure what reading the other side results in (I don't think that this is allowed).
My take on this? Please use an exception if you mean an exception (for C++). If it is too slow (which I consider unlikely), please deal with it some other way (I would suggest a status return).
> "The key takeaway here is that an exception cannot be ignored or forgotten; if we have an exception, it must be dealt with."
but a good way to deal with exceptions (in the very specific case of GUI desktop software, though that works also for server software that handles requests) is to just have a global catch handler at the top of your event loop which logs an error message / displays a message box to your user / does a preemptive backup save of the data in case something is deeply corrupted somewhere and then restarts the event loop.
If exceptions aren't used, the error case has to be handled somehow. The usual approach is to return two results, and callers have to check the error flag somehow. If exceptions are very rarely thrown, then even with their very high overhead the total cost could well be lower than the cost of returning a separate error result and having callers check that result, perhaps by pattern matching.
Exceptions are expensive and I hope no body uses it in the hot path (80%). The compiler has to inject a lot of code to unwind the stack that affects the cache. It is however not expensive for common operations (20%).
There are couple of threads in dforum that discusses this and to have throw attribute [1] and eventually making nothrow as default [2].
I made a website where you can measure these things in both time and instructions. It's not perfect because it's a VM, but at least you can see for yourself the horror show that is the first exception thrown. And also the bloat that exceptions add. I still use exceptions simply because they are part of the language, although sparingly. But I have a gut feeling that the implementation of C++ exceptions in the current compilers, and the ABI, is just subpar. It has to be, because nothing that isn't a context switch should be measured in hundreds of thousands of instructions.
You are just measuring initialization time. The global "C" locale used by printf is statically initialized. With constexpr that could be done in a C++ library if anybody cared. Runtime spent initializing might matter for startup in an embedded system, but they tend not to use cout much.
You could submit a PR constexpr-ing std::cout initialization.
If you want to see Herb Sutter's take on exceptions (and RTTI), I can really recommend this talk where he explores his Result-esque (like in Rust) solution that he wrote a paper about and proposes for standardisation: https://youtu.be/ARYP83yNAWk
Spot the Howler: "it’s always best to catch an exception as close to the throw point as possible".
No. It's fastest to do that, but if you cared about fast, you wouldn't be throwing at all.
Exceptions, by definition, are off of the critical path. If there's a generalization to be made, it's that it's always best to catch them as far from the event as possible, because then you will have few catch clauses. If you have more than a few, you are Doing It Wrong.
It shows a fundamental misunderstanding to worry about how fast throwing an exception is. You legitimately worry about how fast it is not to throw them, and that is generally quite fast indeed.
It's popular because it gives to people what they want to hear. There has been lots of research that gave different results and people just love to ignore.
The overhead also depends on how the compiler implements them
Some compilers use setjump/longjump (sjlj), and then it needs to save all registers somewhere on every try{}. Probably also in every function that calls a destructor. That is very slow, even if no exception is thrown at runtime
I personally have a hard time understanding std::error_code. Specifically I can't figure out when std::generic_category vs std::system_category should be used, and I worry that confusion between the 2 of them will lead to confusing and inconsistent APIs where it's hard to predict what error_code will be returned in what situation.
std::system_category and std::generic_category are generally the same on non-Windows platforms. std::system_category on Windows is any Windows error (HRESULT, GetLastError, etc) and std::generic_category always represents errno.
So for cross-platform APIs that exist on both you'll generally use std::generic_category as the API is usually defined to modify errno in the case of an error. For Windows-only APIs use std::system_category.
Any idea of the cost of the final approach they used for the “success” case? E.g. if the rand errored 1/100, would the cost of returning that union object outweigh the exception?
1. Exceptions are necessary for correctness. This alone solves the question of whether they should be used. Error codes just don't cut it because they only handle expected errors (which is never enough in the real world).
2. Arguing about C++ exception performance is pretty pointless anyway since C++ performance is its (only) strong side. Until Java/C# matches C++ in e.g. cutting edge 3D engines, this question won't even become interesting.
I disagree with both points, in fact with every statement you made. Correctness can be achieved without use of exceptions (whether this is easier or harder to do is up for debate). Next, in some cases correctness may be sacrificed for performance reasons (e.g. solving a traveling salesman problem). I find it easier to properly handle unexpected error codes than unexpected exceptions. C++ has other strong sides beside performance. Java/C# often throw exceptions in very non-exceptional cases (e.g. .Net's File.Delete method).
Nope, there's no practical way to cover all possible cases. There is always a possibility of a totally exceptional situation arising (we're only human). And then your beautiful ole error-cods returnin' function either takes down your whole application (which might result in someone dying while on life support in a hospital) or plods on with a successfull error code (leaving your program in an incorrect state). All because you didn't have a catch-all-exceptions clause.
This is why I find C++'s "nothrow" a joke, by the way. It's not within human power to guarantee that any piece of code never throws exceptions. A gamma ray might flip a bit. You never know. The only practical way to reliability is the Erlang way: be prepared for errors arising anywhere anytime, not rely on fallible notions of what can and cannot cause errors.
There is a difference between preventing a program crash and handling errors. Please don't use catch-all-exceptions clause for either, especially not in hospital machines.
The difference between writing if (status != SUCCESS) and catch (Exception) is that the former will allow the program to actually crash when it should (see panic in Rust). You should not be able to recover from OutOfMemory or GammaRayFlippedABit errors (use ECC memory to prevent that). You should restart the process (or the entire machine) because you can't hope the developer was wise enough to destroy and recreate appropriate amount of state.
The additional problem with exceptions in Java and C# is that many times you don't care about failure of some operations, but the language practically forces you to write catch-all statement to prevent a program crash, so you end up accidentally catching unrecoverable errors.
> there's no practical way to cover all possible cases
This is something that can be enforced by the language, it's just that C doesn't do so. The Zig language does, though. The result is somewhat similar to conventional exceptions. [0]
It would be possible to do the same in C if a convention were used that could be verified by static analysis. I don't know if this has ever been implemented, though.
> And then your beautiful ole error-cods returnin' function either takes down your whole application (which might result in someone dying while on life support in a hospital) or plods on with a successfull error code (leaving your program in an incorrect state).
That's not an accurate account of how critical systems are written.
Plenty are written in C, which has no support for exceptions. Some are written in the SPARK language, a subset of Ada, which essentially forbids the use of exceptions. [1]
> This is why I find C++'s "nothrow" a joke, by the way. It's not within human power to guarantee that any piece of code never throws exceptions.
That's not right. Code written in C is guaranteed never to throw an exception, as the language doesn't support them. Error codes have to be used. Code can still go wrong, and can even produce undefined behaviour, but exceptions are never raised.
I'm not very knowledgeable on the particulars of nothrow and noexcept in C++ though, I have to admit. From a quick glance, it seems that it does provide a hard assurance that a noexcept function can never throw. If a noexcept function tries to throw, the effect is to call std::terminate. [2]
> A gamma ray might flip a bit.
That won't throw an exception, it will just flip a bit. Radiation-hardening is handled by hardware, and perhaps by replication, but not by software. [3]
> The only practical way to reliability is the Erlang way: be prepared for errors arising anywhere anytime, not rely on fallible notions of what can and cannot cause errors.
Again, that certainly isn't the only way. C and SPARK are both used for life-or-death code. No-one will ever write avionics software in Erlang.
Performance overhead, I'm not that worried about - if you're throwing exceptions routinely in the critical path, there's likely a lot else wrong with your code. The crappy part about C++ exceptions is that they also introduce a substantial binary size overhead.
Exceptions make your code faster in the non-exceptional case.
See all those if-branches for error checks that don't exist in your code? That branch-free code is going to run faster than your branchy code. And the code's shorter, because it doesn't have the checks and handling intermixed, which makes it fit better into cache.
This isn't rocket science, I'm amazed that this isn't obvious.