A more efficient way of reading MD trajectory

[njzjz]

In the workflow, we do not need to read every frame of trajectory, but only what we want. So, we should firstly make the following dict to map the frame to the trajectory:

frames_dict = {
  Trajectory0: [23, 56, 78],
  Trajectory1: [22],
  ...
}

Then, reading each trajectory:

for traj, f_idx in frames_dict.items():
    traj.read(f_idx)

For a LAMMPS trajectory or other raw text files, the read should be

def read(self, f_idx: list[int]):
    with open(self.fname) as f:
        for ii, lines in enumerate(itertools.zip_longest(*[f] * self.nlines)):
            if ii not in f_idx:
                continue
            self.process_block(lines)

where nlines is the number of lines in each block, which should be determined in the very beginning. Usually, every frame has the same number of lines.

process_block method should convert a LAMMPS frame to dpdata.

[amcadmus]

This way of loading trajectory should replace the implementation in https://github.com/deepmodeling/dpgen2/blob/2011090d12ba26a1eb3849634883f9ae0b62cc9d/dpgen2/exploration/selector/conf_selector_frame.py#L132-L138

The problem is how. Shall we provide this highly efficient way of reading frames from trajectories to dpdata or directly implement it to dpgen2?

[njzjz]

We should add to dpdata, so others packages which use dpdata will also be benefited.

[njzjz]

Enhance this method:

dpdata/dpdata/lammps/dump.py

Lines 139 to 158 in e8a2fbe

	def load_file(fname, begin = 0, step = 1) :
	lines = []
	buff = []
	cc = -1
	with open(fname) as fp:
	while True:
	line = fp.readline().rstrip('\n')
	if not line :
	if cc >= begin and (cc - begin) % step == 0 :
	lines += buff
	buff = []
	cc += 1
	return lines
	if 'ITEM: TIMESTEP' in line :
	if cc >= begin and (cc - begin) % step == 0 :
	lines += buff
	buff = []
	cc += 1
	if cc >= begin and (cc - begin) % step == 0 :
	buff.append(line)