More surprising is that, even ignoring the allocation cost, other operations on the memory area seem more expensive:

$ ./python -m timeit -s "b=bytearray(500000)" "b[:] = b"
-> python 3.3:
1000 loops, best of 3: 367 usec per loop
-> python 3.2:
10000 loops, best of 3: 185 usec per loop

(note how this is just a dump memcpy)