Much as been written about the nature of time, including philosophical viewpoints named eternalism and presentism. However, a general principle in science is that the best explanation is the simplest one that accounts for all observations. When discussing time travel, three issues are involved in explaining why it is so poorly understood.
First, our animal brains evolved millions of years ago to remember past events, in what sequence events take place, and to anticipate the future. These abilities helped to protect our very distant ancestors from predators and other natural dangers. But this brain structure, necessary to stay alive, misinforms our perception of reality. Our animal brains cause us to see reality as a movie, with a past, present, and future, and with an expectation of events happening in sequences.
Second, in science, the meaning of each word must be defined exactly to convey an accurate meaning. The Britannica Dictionary has 104 different entries for time, making it one of the most poorly defined words in use. When discussing the fourth dimension, the only definition of time that is appropriate is that of Einstein's Theory of Relativity: time is rate of change. Rate of change slows down as gravity increases, by billionths of a second, and speeds up as it diminishes.
Third, the first three dimensions of space-time have the attribute of position, but the fourth dimension, rate of change, does not. We subconsciously associate this attribute to the fourth dimension without being aware that we are making a logical error.
When it comes to time travel, there is no there to go to because position does not exist for rate of change. We are here, in the only part of time that can exist: the immediate now, and in the presence of constant change. Past and future do not exist except as memories and concepts in our animal brains. This very simple view of time is all we need to understand that time travel is a concept, not reality, and has found its best use as entertainment.
While there is no past, the universe has memory, of a sort, in the assembled history of complex molecules, layered sedimentary and igneous rocks, tree rings, and evolution revealed by DNA analysis across species. But that record is only a partial account of the past, and far from complete.
So does time, as rate of change in Einstein's universe, have any relationship to time as the assembled history of complex objects? What if we dropped the word time and used specific phrases instead, such as rate of change and assembled history? Do these two refer to the same aspect of the universe? No. Their relationship is only indirect.
Eliminating the word time from scientific discussions reduces the possibility of improperly combining disparate concepts, and the tendency to subconsciously view time as some sort of overarching supernatural dimension.