Well, it gets a bit complicated because while the 400 and 500 series are of the same generation, the 600 series is a much different GPU architecture.
The cores in the 600 series are not the same cores in the 400/500 series. They are simpler, doing no instruction reordering, and instead of operating at twice the GPU's frequency (the shader clock) the entire GPU has the same clock. So, comparing the 580 and 680:
580: 512 cores, instruction reordering, 1.55GHz speed
680: 1536 cores (3x), no instruction reordering, 1GHz speed (33% slower, but with +50MHz average GPU boost (5%))
Overall: 680 is 3 x .66 = 2x faster, minus some variable amount for the removal of instruction reordering (??%).
The reason for the clock speed decrease is that power consumption increases with the square of the clock frequency (2x freq = 4x power consumption), while adding more shaders is a linear increase (3x shaders = 3x power consumption). This prompted Nvidia to reduce power and heat by adding more shaders but clock them lower.
However, more die space is now needed for the shaders, so the shaders themselves had to give up some logic to make them smaller -- and the logic sacrificed was the instruction reordering. As it only benefits the GPU in some cases and is fairly significant in terms of die space, it was the first shader feature on the chopping block. This pushes the responsibility of generating an ideal instruction stream to the GLSL and OpenCL compiler. This generally doesn't affect graphics much, but the loss of this feature causes OpenCL kernels to execute slower.
Finally, how the GPU cores are arranged in high-level blocks make the 680 the successor to the GEForce 560. The 560 which was a bit weaker at compute because of the 2:3 scheduler:work unit ratio that it used (2 schedulers to 3 blocks of 16 shaders). The 680 has 6 shader blocks (32/block) to 4 schedulers, giving it a 2:3 ratio as well. So sometimes all 6 blocks are running, and sometimes only 4 can run, depending on the workload. However, if you compare the 680 to the 560, it has a 4x shader advantage and ~20-25% clock speed advantage. By comparison, the 580 uses a 2:2 scheduler:shader block ratio.
So, in a nutshell, comparing GPUs of different generations by cores or clock speed isn't quite apples-to-apples, and that's the reason the 680 wins some (GL), loses some (CL).