VMD-L Mailing List

From: John Stone (johns_at_ks.uiuc.edu)
Date: Wed Sep 10 2003 - 16:07:34 CDT

Hi Peng,
  First thing, I'd recommend upgrading to the final version of
VMD 1.8.1 (fixed numerous bugs that were in the ss03 version, which
after all was really just a stopgap release prior to the final VMD 1.8.1)
Beyond that, since you're running on Windows, you are definitely subject
to 32-bit filesystem limitations. The normal C library read/write calls
are all limited to operations on 32-bit files. Any file larger than 2GB
will fail on fseek()/fread()/fwrite() operations past the 2GB mark.
So, its not that you don't have enough memory, its that Windows only has
32-bit file pointers in their implementation of the C library. Unfortunately
This basically means that you'll have to split your trajectory file into
multiple pieces.

The easiest way to do that would be with a text splitting and concatenation
utility since you're using Amber trajectory files. Do you have this large
trajectory file copied onto your PC's local hard drive, or are you accessing
it over a network from some other file server? If its over a network,
you may discover that NO program will let you read past the 2GB mark when
run locally on your Windows machine. You'll have to find a way to split
the file into pieces, once this is done, you can put the correct header
information back on the file so that VMD can read it. You might try using
'ptraj' to do this sort of thing if you can run it on a machine with
64-bit filesystem support. Most utilities you've got on Windows will
fail at the 2GB mark since they typically use the C file I/O routines.

If you use a Sun, SGI, or some other 64-bit machine, you'll avoid all
of these problems because they have 64-bit file I/O routines, and things
like fseek(), ftell(), fread() and fwrite() all work correctly on files
larger than 2GB...

Thanks,
  John Stone
  vmd_at_ks.uiuc.edu

On Wed, Sep 10, 2003 at 04:12:44PM -0400, Peng Tao wrote:
> Dear John Stone,
>
> Thank your replying.
>
> Here are the details:
> ////////////////////////////////////////////////////////////////////////////
> //////
> Microsoft Windows 2000 [Version 5.00.2195]
> vmd > vmd
> Info) VMD for WIN32, version 1.8.1ss03 (May 27, 2003)
> Info) OpenGL renderer: GDI Generic
> Info) Features: STENCIL
> Info) Textures: 2-D (1024x1024)
> Info) Spaceball driver not installed. Spaceball interface disabled.
> Info) No joysticks found. Joystick interface disabled.
> Found 69 plugins or data handlers in directory
> 'C:/Program Files/University of Illinois/VMD/plugins/WIN32/molfile'.
> ////////////////////////////////////////////////////////////////////////////
> //////
>
> The traj file is mdcrd file from the AMBER7 MD simulation. The actual length
> is 2,147,483,647 byte. The system I am simulating contains 32901 atoms.
>
> Yes I am using 32-bit computer, but I didn't try to read in the whole
> trajectory file. But VMD couldn't access any frame after 2684. Do you think
> it is due to 32-bit addressing limit?
>
> Is there any way for me to divide the large mdcrd file into parts? Therefore
> I can avoid such limit?
>
> Thank you very much.
>
> Yours Sincerely,
> Peng Tao
>
> Chemistry Department
> The Ohio State University
>
>
> ----- Original Message -----
> From: "John Stone" <johns_at_ks.uiuc.edu>
> To: "Peng Tao" <ptao_at_chemistry.ohio-state.edu>
> Cc: <vmd-l_at_ks.uiuc.edu>; <vmd_at_ks.uiuc.edu>
> Sent: Wednesday, September 10, 2003 3:56 PM
> Subject: Re: Reading large traj file error or how to divide a large traj
> file into parts?
>
>
> >
> > Hi,
> > You've just discovered the 32-bit addressing limit. :-)
> > Unless you're using a 64-bit computer (i.e. Sun, SGI, IBM RS/6000,
> > Itanium, or Opteron / Athlon64) you are not going to be able to easily
> > load files that are larger than 2GB. You didn't mention which version
> > of VMD you're using nor which platform you're running on, but it
> definitely
> > sounds to me like you're running on a 32-bit machine (or at least a 32-bit
> > version of VMD).
> >
> > If you have a 64-bit Sun, SGI, etc sitting around in your lab, that's
> > going to be the easiest way for you to load the large file. We have
> > 64-bit LFS code in some of the VMD file reader plugins, but I don't
> > think that this has made it into our Gromacs readers quite yet.
> > (I assume that "traj" in this case refers to a Gromacs .trj file?)
> >
> > Thanks,
> > John Stone
> > vmd_at_ks.uiuc.edu
> >
> > On Wed, Sep 10, 2003 at 03:01:22PM -0400, Peng Tao wrote:
> > > Dear VMD users,
> > >
> > > After 2ns simulation (nstlim=2,000,000, ntwx=200, so there should be
> 10,000 frames in the traj file ), the file size is about 2GB. It's so large
> that I couldn't read it in for one time. So I tried to use VMD to read in
> 1000 frames each time and analyze the results, that is, read 0-999 frames,
> deleted them, then read 1000-1999 frames, etc..
> > >
> > > But for 2000-2999 frames VMD stopped at 684. After deleting the read
> frames (2000-2684), I tried to read 2685-2999 frames, but got nothing.
> > >
> > > Then I tried to read one frame from every 10 frames, that is, for
> Frames: First=0, Last=-1, Stride=10. After long time waiting, I only got 269
> frames. It seems to me that VMD only can access the frist 2685 frames of
> this large traj file.
> > >
> > > Is there any way to access other frames? Or can I cud the large traj
> file into parts, therefore I can access the frames those are near to the
> end?
> > >
> > > Thank you very much.
> > >
> > > Have a nice day and happy Moon Festival.
> > >
> > > Yours Sincerely,
> > > Peng Tao
> > >
> > > Chemistry Department
> > > The Ohio State University
> > >
> >
> > --
> > NIH Resource for Macromolecular Modeling and Bioinformatics
> > Beckman Institute for Advanced Science and Technology
> > University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
> > Email: johns_at_ks.uiuc.edu Phone: 217-244-3349
> > WWW: http://www.ks.uiuc.edu/~johns/ Fax: 217-244-6078
> > 

-- 
NIH Resource for Macromolecular Modeling and Bioinformatics
Beckman Institute for Advanced Science and Technology
University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
Email: johns_at_ks.uiuc.edu                 Phone: 217-244-3349              
  WWW: http://www.ks.uiuc.edu/~johns/      Fax: 217-244-6078