That is why voltage is called “potential”. It has the potential, but until the circuit is completed, there is no current.
Not a bad way to think of it. But the science behind the term goes deeper. Please allow me to offer this, if only for its entertainment value:
Consider a positively charged particle sitting somewhere in space. Point "A" is some distance away. Point "B" is further away along the same line. Also along the same line, much farther away (we don't need to know exactly how far), is another positively charged particle. The two positive charges repel each other, so it will require some energy to move the second particle closer to the first particle.
The amount of energy needed to move the second particle from wherever it started to point "B" is called the "potential energy at point B." I will abbreviate it as "PE-B." I will use "PE-A" to designate the "potential energy at point 'A' ." Clearly, PE-A is larger than PE-B, because the second particle would have further to travel, in order to make its way to point "A."
If you subtract PE-A minus PE-B, you get the "potential difference" between the two points. That "potential difference" is the basic meaning of "voltage." That is why we must always speak of voltage as a value taken between two points in a circuit.
The amount of energy needed to move the second particle depends of the amount of charge sitting on that particle. If the charge is higher, the repelling force it feels will be greater. Thus, the parameter "volt" is defined as "joules" (an amount of energy) per "coulomb" (an amount of charge).