Page 1 of 2 12 LastLast
Results 1 to 15 of 16
  1. #1
    Plutonium Lounger
    Join Date
    Dec 2000
    Location
    Sacramento, California, USA
    Posts
    16,775
    Thanks
    0
    Thanked 1 Time in 1 Post

    Is File Completely Written? (Access 97 SR-1)

    We have an application that functions as an import "server". It monitors a folder at intervals and attempts to unzip and import files that have been placed in that folder. The problem we're running into is that it appears the communications program that downloads the files to that folder has not finished writing the file before the import application grabs it and tries to unzip it.

    Does anyone have a suggestion as to how we can determine whether the file has been completely written before the application attempts to use it? We tried placing a read lock on the file using the Open syntax like this:

    Open strFile For Append Lock Read As #intNo

    Unfortunately, that doesn't appear to work, since it succeeds in placing a read lock on the file even though it isn't completely written. That results in a zero length file. Using Read Write rather than Read gives the same result. Help?
    Charlotte

  2. #2
    4 Star Lounger
    Join Date
    Dec 2000
    Location
    Faifax, Virginia, USA
    Posts
    542
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Is File Completely Written? (Access 97 SR-1)

    Assuming that the file can "wait a little bit" before being processed, how about capturing the file names in the folder along with their sizes. Use the list of files to check the folder again say, a minute later, and compare sizes. If the size is the same as the last time, is it reasonable to assume the file d/l is complete?

  3. #3
    Gold Lounger
    Join Date
    Jun 2001
    Location
    Crystal Beach, FL, Florida, USA
    Posts
    3,436
    Thanks
    1
    Thanked 34 Times in 34 Posts

    Re: Is File Completely Written? (Access 97 SR-1)

    My solution would have been similar to Peter's. I was just going to suggest a time-delay once a file has been found, a delay that would likely allow the file to finish writing. Peter's is more bullet-proof.
    Mark Liquorman
    See my website for Tips & Downloads and for my Liquorman Utilities.

  4. #4
    Plutonium Lounger
    Join Date
    Dec 2000
    Location
    Sacramento, California, USA
    Posts
    16,775
    Thanks
    0
    Thanked 1 Time in 1 Post

    Re: Is File Completely Written? (Access 97 SR-1)

    The problem is determining when a file has actually been completely written. We thought of the repetitive approach, but it's a bit vague. Our latest idea is to write a dummy file with the same name but a different extension after the real file has been created. The monitoring program would look for the dummy file and import a real file with the same filename only if it found the dummy.

    Apparently this is an even bigger problem with FTP file transfers than with dial-up and network transfers, so we're going to have to figure out the best all-around solution we can come up with.
    Charlotte

  5. #5
    Platinum Lounger
    Join Date
    Dec 2001
    Location
    Melbourne, Australia
    Posts
    4,594
    Thanks
    0
    Thanked 27 Times in 27 Posts

    Re: Is File Completely Written? (Access 97 SR-1)

    We had a similar problem, and solved it as follows.
    Say you have data files with filenames like filename_d.txt, we introduced a trigger file filename_t.txt which contains nothing at all. The data file can be copied from where ever and the import programs would just look for the trigger files. When it found any trigger files it would correctly assume that the data file was there waiting in its entirety. So the communications program would :
    a) send the data file (filename_d.txt)
    [img]/forums/images/smilies/cool.gif[/img] after sending the data file would send a trigger file of 0 bytes (filename_t.txt).
    This solved our problems.
    Pat

  6. #6
    Gold Lounger
    Join Date
    Jun 2001
    Location
    Crystal Beach, FL, Florida, USA
    Posts
    3,436
    Thanks
    1
    Thanked 34 Times in 34 Posts

    Re: Is File Completely Written? (Access 97 SR-1)

    Apparently you file transfer times are rather lengthy, and not with a standard duration. (I had assumed a relatively short duration time.) Given that, I like the use of the second file as the trigger.
    Mark Liquorman
    See my website for Tips & Downloads and for my Liquorman Utilities.

  7. #7
    Silver Lounger GARYPSWANSON's Avatar
    Join Date
    Aug 2001
    Location
    Frederick, Maryland, USA
    Posts
    1,788
    Thanks
    0
    Thanked 2 Times in 2 Posts

    Re: Is File Completely Written? (Access 97 SR-1)

    You could import the files to a temporary storage folder and then run a program that copies from the temporary storage folder at various times during the day to the processing folder that will unzip and/or process the fully loaded files. The program should be unable to copy the files to the processing folder if they are still being imported. Of course, it would be better to run a test on the files in the temporary storage folder to ensure that the download is complete prior to copying or moving them to the processing folder.

    This technique may work with some of the other ideas presented.
    Regards,

    Gary
    (It's been a while!)

  8. #8
    Plutonium Lounger
    Join Date
    Dec 2000
    Location
    Sacramento, California, USA
    Posts
    16,775
    Thanks
    0
    Thanked 1 Time in 1 Post

    Re: Is File Completely Written? (Access 97 SR-1)

    It isn't that they're so lengthy, although we have no way of knowing how large they will be, but that they are coming in through a communications program, so the entire file isn't written all at once, it's written as it is received over dial up or FTP or WAN or whatever. If the file takes a second or two to write and ten of them come across, pausing might work for the first nine and interfere with the tenth. <img src=/S/shrug.gif border=0 alt=shrug width=39 height=15> The solution has to work on all files regardless of how the files are received.
    Charlotte

  9. #9
    4 Star Lounger
    Join Date
    Dec 2000
    Location
    Faifax, Virginia, USA
    Posts
    542
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Is File Completely Written? (Access 97 SR-1)

    I like the trigger-file idea better than the nebulous period of waiting that i suggested. But it requires a change to the sender and the receiver.

  10. #10
    WS Lounge VIP rory's Avatar
    Join Date
    Dec 2000
    Location
    Burwash, East Sussex, United Kingdom
    Posts
    6,280
    Thanks
    3
    Thanked 191 Times in 177 Posts

    Re: Is File Completely Written? (Access 97 SR-1)

    Charlotte,
    This is (as usual) probably off-track but, if you use the API call LZOpenFile using the OF_SHARE_DENY_WRITE parameter, then it should fail, and hopefully you could trap that? According to AllAPI, it "Opens the file and denies other processes write access to the file. LZOpenFile fails if the file has been opened in compatibility mode or has been opened for write access by any other process."
    I don't have the wherewithal to test it but thought it might be worth looking at.
    Regards,
    Rory

    Microsoft MVP - Excel

  11. #11
    Plutonium Lounger
    Join Date
    Dec 2000
    Location
    Sacramento, California, USA
    Posts
    16,775
    Thanks
    0
    Thanked 1 Time in 1 Post

    Re: Is File Completely Written? (Access 97 SR-1)

    Thanks, Rory, I'll take a look at it. That might be what we need.
    Charlotte

  12. #12
    Plutonium Lounger
    Join Date
    Dec 2000
    Location
    Sacramento, California, USA
    Posts
    16,775
    Thanks
    0
    Thanked 1 Time in 1 Post

    Re: Is File Completely Written? (Access 97 SR-1)

    Rory,

    We couldn't make it work. We ran into the same problem as with the Open method. Actually, what seems to work best is simply trying to copy the file. That returns a permission denied error if the file is currently being written to. <img src=/S/shrug.gif border=0 alt=shrug width=39 height=15>
    Charlotte

  13. #13
    Silver Lounger
    Join Date
    Jun 2001
    Location
    Niagara Falls, New York, USA
    Posts
    1,878
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Is File Completely Written? (Access 97 SR-1)

    Hi Charlotte

    Had similar requirement years ago, however not with Access, off the top went somethign like this:

    1. Have all incoming files go into F:FileIn

    2. Have Access program copy F:FileIn*.* to F:FileInProcess (zreo length files don

  14. #14
    Plutonium Lounger
    Join Date
    Dec 2000
    Location
    Sacramento, California, USA
    Posts
    16,775
    Thanks
    0
    Thanked 1 Time in 1 Post

    SOLVED: File Completely Written (Access 97 SR-1)

    It isn't a zero-length file issue in this case, but the fact that the file is locked by the process writing it keeps it from being copied. In that case, we ignore it until the next time around.

    What we do is create an array of path and filenames for the files in the directory being monitored. Then we call a routine that tries to copy the each file into a Temp folder.

    If it can, it kills the copy in the temp folder and processes the file, which involves unzipping the file and importing data from a series of text files into tables in the database. Then it moves the file it processed into a folder called IMPORTED.

    If it can't copy the file to the Temp folder, it moves on to the next file in the array and tries it. Any files left in the folder because they are still being written will get picked up on the next timed check of the folder.
    Charlotte

  15. #15
    4 Star Lounger
    Join Date
    Dec 2000
    Location
    Faifax, Virginia, USA
    Posts
    542
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: SOLVED: File Completely Written (Access 97 SR-1)

    cool! <img src=/S/cool.gif border=0 alt=cool width=15 height=15>

    What unzip component are you using?

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •