Did you also try the 16.5 MHz version? If I look at the F vs. OSCCAL drawing on p. 607 of the datasheet* with split OSCCAL and you use the default starting value of 128 you're in the upper half that starts with 6..6.5 MHz (-> clock frequency 12..13 MHz) and grows.
[I guess you mean figure 33-39 of the datasheet. At least that's what I'm referring to.]
Where do you get that doubling of the frequency? The internal RC oscillator of the ATmegas is specified for 8 MHz. The with the OSCCAL register one can push it to around 13 MHz, but that's about it.
In general: micronucleus has some code that tries to calibrate the OSCCAL on the frame time of the USB bus. I can see it speeding up the MCU when it is connected to the USB, so it does something. But it doesn't seem to be good enough.
P.S. I didn't find any mention of using V-USB on the ATmega328p with the internal oscillator, so I guess that just doesn't work.