Will a drop of olive oil in my eye help relieve dry eye?

I have a condition called dry eye that sometimes, during my sleep, results in abrasion to the surface of the cornea. I use an ointment on my eyes recommended by my doctor before bed every night, but am wondering if it would be safe to substitute olive oil if I ever ran out of ointment and couldn't get to the store before bed.