In Desperate Need of Help for Creating a Macro Tool Set

Hello fellow Stack friends,

I understand this site is more of a specific Q/A, so I hope this post doesn’t bother anyone.

I hope this is the right place to ask for help. I need to create a macro tool set to read .csv files. I was advised to learn python to best achieve this. A friend sent me some code to go off of and I’m having a bit of trouble understanding this. Over the past three weeks, I took the Udemy course – Automate the Boring Stuff with Python and felt like it had great information but trying to understanding someone’s code is a bit of a struggle. I’ve also been watching some of Corey Shafer’s Youtube videos to educate me on different libraries to import. (Pandas, RE, matplotlib, argparse, openpyxl, os, sys)

Example picture from excel

This is what I need my code to do: (This is the best pseudocode I (a newb) could come up with)

  • Read in .csv file

  • Count how many rows & columns and print out on console to confirm

  • Search .csv file for specific text (headers) and print out into a new .csv file

  • This will consist of searching for a few unique text (headers) and printing them out onto the first two rows (code below shows: def getdata function has a custom made library… I’m getting some of the data to show up when I create a new .csv file so I think I’m okay there) (This is where I’m having trouble…)

Example: One Cell will look like – Revision Number: ABCD

Row 1 Displays: Revision Number:

Row [2] Displays: ABCD

note: I hope this is clear, but the whole text is in one cell. So I’m not sure how to break up the cell after the (:) I read you split up the cell by adding (‘,’)

note2: In the code, I was able to split the first cell after the (:) but the code stops working after the second searched text.

note3: the searched text for this part is just to create headers of the main data. I’m going to need to run this program to compare and display data on hundreds of different .csv files. (I hope to collect this data and compare to each new file I’m pulling from)

  • Search .csv file for specific text (ie. sensors/components) and print out into a new sheet (for each sensor/component) of the new .csv file

  • To do this, I’ll need to read/find specific text (ie. Verify sensor ___), print out into a new sheet and also print the following list of data being tested from the sensors/components

  • The data after finding specified text typically consists of 4-5 rows & 10-20 columns of data

  • The data also has headers for max & min values that I’ll need to organize correctly

  • After I have each sensor separated between each new sheet with its corresponding data, I’ll need to generate graphs showing if these sensors/components fall within its threshold limits.

  • I was also shown how you can use python to create/generate a shaded region displaying its previous data collected from the previous .csv files and see if the new data for the sensors/components still fall within those ranges.

  • Last notes: A friend showed and provided me with some code to go off of. (provided below) They said they created a batch file where all they need to do is call the file they want to search and it will automatically pull all of its data then screenshot images of the graphs for EACH sensor/component for them to simply go through and compare. I thought it was so awesome and I hope to be able to achieve this. I’ll try to provide a screenshot if my friend is willing to share it.

Code provided by friend:

note: He created a custom library to call any keyword: (ie: ‘p’ : ‘Test Procedure Revision Letter: ‘ if he only wanted to find and print that header)

note2: Some of the code doesn’t work.

Example: ‘A’ : ‘Number of Fails: ‘, ‘F’ : ‘Number of Alerts: ‘,

breaks because the file I’m pulling from has numerous text displaying Number of Fails/Alerts after each sensor. I’m trying to figure out how to code this as well to display the Fails/Alerts in each new sheet for each sensor/component

I really hope someone on here can help me with this. I’ve been going at this for weeks, either trying to understand their code or start from scratch and write my own code.

#!/usr/bin/env python #  PLOT DATA FROM TEST LOG CSV FILE #  Command example with a full string in the above order would be: # ./TEXT.csv -H -r pueuifOBcmtDTCAF  import argparse  # Command Line Parser Setup  parser = argparse.ArgumentParser(description='Create a report from UMATS data')  parser.add_argument('datafiles',\  metavar='DATAFILES', nargs='*',\  type=argparse.FileType('r'),\  help='UMATS data file in CSV format')  parser.add_argument('-r','--report', help='string of characters defining the report format', required=True)  parser.add_argument('-H','--header', help='print out the header',  required=False, action='store_const', const=1)  argz = parser.parse_args()  args = vars(argz)  def readfile(this_fp) :  this_filedata = this_fp.readlines()    return this_filedata  def getdata(that_filedata, that_item) :  count = 0  item_dict = {      'p' : 'Test Procedure Revision Letter: ',     's' : 'UUT Software Version Revision Letter: ',     'u' : 'UUT Software Media Revision Letter: ',      'e' : 'Exec  Software Media Revision Letter: ',     'U' : 'UMATS Serial Number: ',                     'i' : 'ITA Serial Number: ',                       'f' : 'Fixture Serial Number: ',                   'O' : 'UUT MSO Oper: ',                            'B' : 'Badge Number: ',                            'c' : 'Test Condition: ',                          'm' : 'Module Type: ',                             't' : 'Test Type: ',                                'D' : 'Start Date: ',                              'T' : 'Start Time: ',                              'C' : 'Comments: ',                                'A' : 'Number of Fails: ',                   'F' : 'Number of Alerts: ', }  for line in that_filedata :      if item_dict[that_item] in line :      if ((that_item == 'A') or (that_item == 'F')) :      that_itemstring = line.strip(item_dict[that_item])      that_itemstring = that_itemstring.strip('\r\n')      that_itemint = int(that_itemstring)      count = count + that_itemint      that_itemstring = str(count)  else:      that_itemstring = line.strip(item_dict[that_item])      that_itemstring = that_itemstring.strip('\r\n')  break  return that_itemstring  # Vars  loop_cnt = 0  header_dict = {      'p' : 'TP',     's' : 'UUT SW Rev',     'u' : 'UUT SW',     'e' : 'Exec SW',     'U' : 'UMATS',         'i' : 'ITA',           'f' : 'FXT',       'O' : 'OP',                'B' : 'Badge',                'c' : 'Cond',              'm' : 'Module',                 't' : 'Type',                   'D' : 'Date',                  'T' : 'Time',                  'C' : 'Comment',                    'A' : 'Fails',             'F' : 'Alerts',     'S' : 'Date-Time',        }  # Main  report_string = args['report']  for fp in argz.datafiles :      header = ''     item_report = ''     charlist = []     itemlist = []      filedata = readfile(fp)  for char in report_string :      if char == 'S' :          charlist.append(char)          date = getdata(filedata, 'D')          time = getdata(filedata, 'T')          datetime = date + ' ' + time          itemlist.append(datetime)         else :          charlist.append(char)          itemlist.append(getdata(filedata, char))  for char in charlist :      header = header + header_dict[char] + ","      i=0  #itemlist.append("test")  for char in itemlist :      item_report = item_report + itemlist[i] + ","      i=i+1      if ((loop_cnt == 0) and (args['header'])) : print(header)      print((item_report))      loop_cnt = loop_cnt + 1 

Thank you

Add Comment
0 Answer(s)

Your Answer

By posting your answer, you agree to the privacy policy and terms of service.