Much as been written about the nature of time, including philosophical viewpoints named eternalism and presentism. However, a general principle in science is that the best explanation is the simplest one that accounts for all observations. When discussing time travel, three issues are involved in explaining why it is so poorly understood.
First, our animal brains evolved millions of years ago to remember past events, in what sequence events take place, and to anticipate the future. These abilities helped to protect our very distant ancestors from predators and other natural dangers. But this brain structure, necessary to stay alive, misinforms our perception of reality. Our animal brains cause us to see reality as a movie, with a past, present, and future, and with an expectation of events happening in sequences.
Second, in science, the meaning of each word must be defined exactly to convey an accurate meaning. The Britannica Dictionary has 104 different entries for time, making it one of the most poorly defined words in use. When discussing the fourth dimension, the only definition of time that is appropriate is that of Einstein's Theory of Relativity: time is rate of change. Rate of change slows down as gravity and acceleration increase, and speeds up as they diminish. For example, the difference in rate of change between sea level and the top of Mount Everest is 22 millionths of a second.
Third, the first three dimensions of space-time have the attribute of position, but the fourth dimension, rate of change, does not. We subconsciously associate position to the fourth dimension without being aware that we are making a logical error. This includes the "arrow of time" concept, which is only an illusion. Our animal brains remember events happening in sequence, and we think of this sequence as "forward". But forward and backward in time are not attributes of rate of change and have no meaning in physical reality.
When it comes to time travel, there is no there to go to because position does not exist for rate of change. We are here, in the only part of time that can exist: the immediate now, and in the presence of constant change. Past and future do not exist except as memories and concepts in our animal brains. This very simple view of time is all we need to understand that time travel is a concept, not reality, and has found its best use as entertainment.
While there is no past, the universe has memory, of a sort, in the assembled history of complex molecules, layered sedimentary and igneous rocks, tree rings, and evolution revealed by DNA analysis across species. But that record is only a partial account of the past, and far from complete.
So does time, as rate of change in Einstein's universe, have any relationship to time as the assembled history of complex objects? What if we dropped the word time and used specific phrases instead, such as rate of change and assembled history? Do these two refer to the same aspect of the universe? No. Their relationship is only indirect.
Eliminating the word time from scientific discussions reduces the possibility of improperly combining disparate concepts, and the tendency to subconsciously view time as some sort of overarching supernatural dimension.