SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 10 of 10
  1. #1
    Senior Member
    Join Date
    May 2001
    Posts
    378
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Comparing two files, certain column

    Hello all,
    I have two large files which I need to compare. The issue here is that these files are rather large files. They contain around 75,000 rows, way beyond the data limit of WR's data table ability.
    What I need to do is compare a single column in the 2 files. The problem is that I do not have an idea of how to approach this.
    WR's data table row limit is at 16,383 rows, accrding to WR's readme file.
    MS Excel is also limited to 65,536 rows.
    What would you recommend me to do here?
    Maybe I should involve a 3rd party tools here?
    If so, how can I make the connection between the tool & WR so that I can report all the mismatches?

    Thanks.
    If it's working... don't touch it!

  2. #2
    Moderator JakeBrake's Avatar
    Join Date
    Dec 2000
    Location
    St. Louis - Year 2025
    Posts
    15,609
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Comparing two files, certain column

    Export the appropriate columns to flat files - sorting as necessary prior to export.

    Use either the compare function or write your own function to read and compare from the expected result and the actual result files.

  3. #3
    Senior Member
    Join Date
    May 2001
    Posts
    378
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Comparing two files, certain column

    Hello again,
    I do not know how to export these columns in specific.
    I tried opening these files using file_open() and ddt_open() they both return a sharing violation error code (-10041) which is not true since they are not shared.
    I think that WR simply cannot cope with these large files.
    Any idea how I can break it to smaller portions?

    Thanks again.
    If it's working... don't touch it!

  4. #4
    Moderator JakeBrake's Avatar
    Join Date
    Dec 2000
    Location
    St. Louis - Year 2025
    Posts
    15,609
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Comparing two files, certain column

    Save a copy of the current spreadsheet.
    In any of the copies, delete all unnecessary columns.
    Sort as necessary.
    Use the Excel "Save As" function.
    Save as a delimited text file.
    WR can cope with these.

  5. #5
    Junior Member
    Join Date
    Apr 2003
    Location
    Minneapolis, MN, USA
    Posts
    21
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Comparing two files, certain column

    If you are able to save the one column into text documents, you could then use WinDiff to determine any differences. A slick way of doing this in Excel is to save as XML file... Also, you could make multiple Excel files each files containing 10,000 rows.

    Hope that helps.

  6. #6
    Senior Member
    Join Date
    May 2001
    Posts
    378
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Comparing two files, certain column

    Hi again,
    Thanks for the replies.
    I have tried using Windiff. It seems to work ok. The file is rather large so that makes me have to go over the WinDiff results and start checking a huge number of rows on that specific column. Very stressfull for me and my eyes...
    I want to make comparisons on the specific column values and report all mismatches using the regular tl_step().
    Regarding what JPensyl suggested, it is an option, but, it makes me have to stop the test run, in order to edit the excel file for the first 65,536 rows do what I do, and then continue with the next portion, so this is not really what I wanted from this automation process.
    Please, any other ideas will be appreciated.

    Lior
    If it's working... don't touch it!

  7. #7
    Senior Member
    Join Date
    May 2001
    Location
    Ankh-Morpork
    Posts
    2,882
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Comparing two files, certain column

    Lior, are these large files flat ASCII text files, and if so, are the columns fixed-width, tab/comma-separated, or what?

    My natural inclination would be to approach this via a Perl script of some sort as Perl is quite adept at manipulating and parsing text files, and could likely do it much faster than WinRunner could (assuming WR was even able to handle these files). With Perl (or similar language/utility) you could read just the column of interest into a new file, then use any file comparison tool to compare the two outputs.
    web site | [url=http://www.ebookworm.us/[/url]

    [i]...Sound trumpets! Every trumpet in the host! / Sixty thousand, on these words, sound, so high the mountains sound, and the valleys resound.</i] (The Song of Roland)

  8. #8
    Senior Member
    Join Date
    May 2001
    Posts
    378
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Comparing two files, certain column

    Hello Charles.
    Now this is something refreshing...
    I never thought of taking this approach. Simply because, for starters, I do not know PERL. I do not know whether it will be difficult to script such a task. Also, what about reports of mismatches, how would I do that? I assume that running the script via WR is not the issue. Where can I read (not a Google search but rather short and to the point article) about Perl and how to script such a thing please...?
    Oh, BTW, I tried using Beyond Compare but I still do not know how to make it compare only one column.
    And the files are tab dilimted and contain ~75,000 rows.

    Thanks again,
    Lior
    If it's working... don't touch it!

  9. #9
    Senior Member
    Join Date
    May 2001
    Location
    Ankh-Morpork
    Posts
    2,882
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Comparing two files, certain column

    Take a look at this Perl script which will compare the same column in two files, printing the line number and the actual contents whenever it finds a mismatch. I just "whipped this up" and have not rigorously tested it, but it might give you a good starting point. Note that I haven't included logic for things like the files not having the same number of lines, so I'm not sure what might happen in such situations.

    Hopefully this will give you some ideas. To run it on Windows you will probably want to go to www.activeperl.com and download/install Activestate Perl (it's free). Save the Perl script file with a .pl extension, then give it a try. You could run it from WinRunner with something like...
    </font><blockquote><font size="1" face="Verdana, Arial, Helvetica">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;">dos_system(&quot;c:\\some_dir\\compare _column.pl c:\\some_dir\\file1 &quot; &amp;
    &quot;c:\\some_dir\\file2 2 \&quot;\\t\&quot; &gt; c:\\diffs.txt&quot;</pre><hr /></blockquote><font size="2" face="Verdana, Arial, Helvetica">...then look at the contents of c:\diffs.txt for any differences.
    web site | [url=http://www.ebookworm.us/[/url]

    [i]...Sound trumpets! Every trumpet in the host! / Sixty thousand, on these words, sound, so high the mountains sound, and the valleys resound.&lt;/i] (The Song of Roland)

  10. #10
    Junior Member
    Join Date
    Aug 2003
    Location
    Canada
    Posts
    1
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Comparing two files, certain column

    Give a try with SlickEdit Diff...

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 10.71%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 06:31 AM.

Copyright BetaSoft Inc.