Writing question?

I've been working on a series of teenage fantasy books for a few years but it's really frustrating because some people say there are small points that sound like either Harry Potter or Chronicles of Narnia. Like I have such things as a magical school in one book and then I have a old farm shed that is a vortex that leads to a different time. Now, if I try to get my series published, do you think these small points could cause lots of troubles for me? Should I just stop writing them?