I believe no such caveat is necessary.

While Warlock slots refresh every short rest, this wouldn't change how the Pearl or Gem worked. (A Pearl could still only be used once until next dawn. A Gem can only be refilled or "imbued" once until next dawn.)

It's not like the pearl gives the Warlock a new, third, spell slot, that takes on life of its own (starts to recharge every short rest).

A pearl can be used once per day to restore a Warlock's spell slot between short rests.
A gem can be used during the day, then imbued, then used again that same day. (But that only holds true for the first of a series of days: on the second and every subsequent day, you can imbue it and use it once)

Apologies if this wasn't what you were talking about.