When running my UpdatePSData.bat file, in each of my quote files, I am getting duplicate entries for Friday the 13th.
Example:
2026-03-13,48.830002,CU.TO
2026-03-13,48.830002,CU.TO
This causes the following return errors in psCheckFiles.txt
Quotes.csv. Date+Symbol should be unique - ERROR. Duplicate records:
I deleted the quotes.csv and restored the quotes folder from a backup I had on Feb 21 and ran the Update script again, and once again I received duplicates for Friday the 13th.
I then went to Yahoo, and sure enough, there are duplicate entries showing in the Yahoo historical data on their website.
Any suggestions?
Comments
I will investigate and will post here when I know solution.
Update script:
Scripts\GetQuotes-Yahoo.ps1 - updated code fixes the issue if Yahoo provides duplicate quotes.
Additional scripts:
Scripts\deduplicate-currExch.ps1
Scripts\deduplicate-quotes.ps1
deduplicate.bat
If you already have duplicate issues, please download updated scripts, copy these 4 files to the corresponding folders (note that .bat files go to the folder above Scripts), and then run deduplicate.bat.
Your fix worked perfectly! As always, your work is greatly appreciated.
Sorry about this - I updated GetExcelSrdData script with version provided by another user here. Old script version was using OleDB to read data from Excel, new version is using PowerShell ImportExcel module.
To solve this, you will have to do import of the model manually.
Can you please follow steps bellow and
Step 1: Start a Command Prompt as Administrator
Open the Windows Start menu, search for cmd, right-click Command Prompt, and choose Run as administrator.
Start Command Prompt as administrator
Administrative privileges may be required depending on how your system is configured and which execution-policy scope you use.
Then, in the Command Prompt window, type PowerShell and press Enter. This will start PowerShell.
Step 2: Run the Required PowerShell Commands
From the command prompt or PowerShell window, check the current policy run 2 statements:
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass -Force;
Import-Module ImportExcel -ErrorAction Stop;
Step 3: Exit from PowerShell and Cmd
Type the following command:
exit
This will exit PowerShell. Then type the same command again to exit Command Prompt.
Please let me know if that fixes your issue.
Can you please try this command instead:
In case helpful, I had to acknowledge a couple items after that command:
Now getting a new error:
Can you please post here screenshot of your source.xlsx file, Transactions tab, just a Date column. Please first sort transactions table by date, and then please post a screenshot of just dates around date 2014-01-31
Your data looks good. Can you please re-download scripts and extract them to temporary folder. Then confirm that script GetExcelSrcData.ps1 has todays updated date and copy that script over to your script location and re-run your script.
I added much more logic to handle dates.
Please let me know if that fixed your issue.
This code if beeing fixed not by me, but by ChatGPT. ChatGPT keeps asking me to provide row transactions data, as it suspects 2 possible scenarios:
- $row.Date is actually a string with hidden characters, or
- the column has mixed Excel storage types across rows.
For now, I uploded updated script. Can you please download it again, confirm that now the date for the script is later than 8:41am (your screenshot) and try again?
I don't really know what I'm doing, but I tried changing all the transactions I have with date 2014-01-31 to 2014-01-30, and re-ran the updates.
The error changed slightly - not sure if this is helpful:
But, before you send it, could you copy that workbook to another file, then from there in Transactions tab delete all values from TransType, TransSubType, SymbolName, Qty, Price, Fee, ExchRate, CostBasisOverrride columns - these columns should be empty (to protect your data privacy). I just need all your transactions records with just Date column.