Intraday not working!!!

trishaan

What am I meant to change in this?????



# 2019-Feb-25. Created by Maxim T.
$scriptPath = Split-Path -parent $MyInvocation.MyCommand.Definition;
. ($scriptPath + "\psFunctions.ps1"); # Adding script with reusable functions
. ($scriptPath + "\psSetVariables.ps1"); # Adding script to assign values to common variables

$logFile = $scriptPath + "\Log\" + $MyInvocation.MyCommand.Name.Replace(".ps1",".txt");
(Get-Date).ToString("HH:mm:ss") + " --- Starting script " + $MyInvocation.MyCommand.Name | Out-File $logFile -Encoding OEM; # starting logging to file.
$logSummary = (Get-Date).ToString("HH:mm:ss") + " Script: Yahoo Intraday".PadRight(28);
$quoteIDFile = $dataRootFolder + "\QuotesIntraDay\YahooIntraday.txt"; if (Test-Path $quoteIDFile) { Remove-Item $quoteIDFile;} # Removing intraday file before each load

$symbolList = @();
$listStart = $config.IndexOf(""); $listEnd = $config.IndexOf("");
if ($listStart -eq -1 -or $listEnd -eq -1 -or $listStart+1 -ge $listEnd) {" Symbol list is empty. " | Out-File $logFile -Encoding OEM -Append; exit(1);}
$symbolList += @($config | Select-Object -Index(($listStart+1)..($listEnd-1))); #list of symbols we will work on

$symbolList = @($symbolList | ?{$a=$_.split(","); ($a[3] -eq $null -or $a[3] -eq "" -or $a[3] -eq "Y") -and ($a[2] -eq "" -or $a[2] -eq $null)}); # by default if value is empty or y, then get intra-day quotes. Also MaxDate for symbol should not be specified
"Symbol count: " + $symbolList.count + ". MinDate: $minDate" | Out-File $logFile -Encoding OEM -Append;

$urlBase = "https://finance.yahoo.com/quote/@@Symbol@@"; $mths = @("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec");

ForEach($sLine in $symbolList) { # For each symbol
$ret = GetSymbolInfo $sLine $quotesFolder $minDate; $symbol = $ret[0]; $nextDate = $ret[1]; $symbolMaxDate = $ret[2]; $symbolMaxDateAdj = $ret[3]; $symbolQuoteFile = $ret[4]; $lastQuote = $ret[5];
"Symbol: " + $symbol.PadLeft(12) + ". Next date: $nextDate. Quote file: $symbolQuoteFile" | Out-File $logFile -Encoding OEM -Append;
if ($nextDate -gt $symbolMaxDate) { " We already have data up to maximum configuration date of $symbolMaxDate. Will not request new data for this symbol." | Out-File $logFile -Encoding OEM -Append; continue; }

# ========== Get latest quote from website - Start ==============
@Symbol@@",$symbol);
" Requesting url: " + $url | Out-File $logFile -Encoding OEM -Append; $webpage = ""; $reqRows=0;
$wc = new-object system.net.WebClient; $reqCount++;
try {$webpage = $wc.DownloadData($url); } # Get page from url. This page will contain all quotes for single Symbol
catch { $reqFailed++; " " + $symbol + " - Not Found (web err) `r`n" | Out-File $logFile -Encoding OEM -Append; continue; } # if attempt to get webpage failed go to next symbol

$quotesTxt = [System.Text.Encoding]::ASCII.GetString($webpage); $quotesTxt = $quotesTxt.Replace("???",""); # This variable now contains downloaded quotes

# ---------------------------------------------
# Identifying price
# ---------------------------------------------
#Price is in this string: 174.96
$startStr = ""; $endStr = "";
#Price is in this string: data-reactid=`"31`">75.73
$startStr = ""; $endStr = ""; ## <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< This will need to be changed every time Yahoo changes where price is located



if ($quotesTxt.Contains($startStr)) {$quotesTxt = $quotesTxt.Substring($quotesTxt.IndexOf($startStr) + $startStr.Length,500)} # This now includes price and date
else {$reqFailed++; " ** For Symbol: " + $symbol + " new quotes not found (there is no '$startStr' in html that came back)`r`n" | Out-File $logFile -Encoding OEM -Append; continue; } # if that symbol has no quotes, then record errors

if (!$quotesTxt.Contains($endStr)) {$reqFailed++; " ** For Symbol: " + $symbol + " new quotes not found (there is no '$endStr' in html that came back)`r`n" | Out-File $logFile -Encoding OEM -Append; continue; }# if that symbol has no quotes, then record errors
$priceTxt = $quotesTxt.Substring(0,$quotesTxt.IndexOf($endStr)).Replace(",","");
" Found price: $priceTxt" | Out-File $logFile -Encoding OEM -Append;
$reqSucceed++;
# Date: >As of 1:48PM EST. Market open. # or date: >At close: 4:00PM EST # At close: 4:00PM EST

$date = (Get-Date).ToString("yyyy-MM-dd"); # Assuming that always returns todays data. If no, you need to disable intraday requests for those symbols
$dayOfWeek = ([datetime]::ParseExact($date,"yyyy-MM-dd",$null)).DayofWeek.ToString().Substring(0,3)
if ($dayOfWeek -eq "Sat") {$date = (Get-Date).AddDays(-1).ToString("yyyy-MM-dd");}
if ($dayOfWeek -eq "Sun") {$date = (Get-Date).AddDays(-2).ToString("yyyy-MM-dd");}

if($date -ge $nextDate) { # If quote date is after or equal to date we already have
$date + "," + $priceTxt + "," + $symbol | Out-File $quoteIDFile -Encoding OEM -Append; $reqRowsT++;
}
}

$duration = (NEW-TIMESPAN -Start $startTime -End (Get-Date)).TotalSeconds.ToString("#,##0") + " sec.";
(Get-Date).ToString("HH:mm:ss") + " --- Finished. Quotes Requested/Succeed/Failed/Rows: $reqCount/$reqSucceed/$reqFailed/$reqRowsT. Duration: $duration`r`n" | Out-File $logFile -Encoding OEM -append;
$logSummary + ". Quotes Requested/Succeed/Failed/Rows: $reqCount/$reqSucceed/$reqFailed/$reqRowsT. Duration: $duration";

VidasMatelis

For next line after: #Price is in this string
Please try changing that line to:

$startStr = "active=`"`" data-reactid=`"29`">"; $endStr = "";

trishaan

Is this correct???
It is still not showing intraday.



# 2019-Feb-25. Created by Maxim T.
$scriptPath = Split-Path -parent $MyInvocation.MyCommand.Definition;
. ($scriptPath + "\psFunctions.ps1"); # Adding script with reusable functions
. ($scriptPath + "\psSetVariables.ps1"); # Adding script to assign values to common variables

$logFile = $scriptPath + "\Log\" + $MyInvocation.MyCommand.Name.Replace(".ps1",".txt");
(Get-Date).ToString("HH:mm:ss") + " --- Starting script " + $MyInvocation.MyCommand.Name | Out-File $logFile -Encoding OEM; # starting logging to file.
$logSummary = (Get-Date).ToString("HH:mm:ss") + " Script: Yahoo Intraday".PadRight(28);
$quoteIDFile = $dataRootFolder + "\QuotesIntraDay\YahooIntraday.txt"; if (Test-Path $quoteIDFile) { Remove-Item $quoteIDFile;} # Removing intraday file before each load

$symbolList = @();
$listStart = $config.IndexOf(""); $listEnd = $config.IndexOf("");
if ($listStart -eq -1 -or $listEnd -eq -1 -or $listStart+1 -ge $listEnd) {" Symbol list is empty. " | Out-File $logFile -Encoding OEM -Append; exit(1);}
$symbolList += @($config | Select-Object -Index(($listStart+1)..($listEnd-1))); #list of symbols we will work on

$symbolList = @($symbolList | ?{$a=$_.split(","); ($a[3] -eq $null -or $a[3] -eq "" -or $a[3] -eq "Y") -and ($a[2] -eq "" -or $a[2] -eq $null)}); # by default if value is empty or y, then get intra-day quotes. Also MaxDate for symbol should not be specified
"Symbol count: " + $symbolList.count + ". MinDate: $minDate" | Out-File $logFile -Encoding OEM -Append;

$urlBase = "https://finance.yahoo.com/quote/@@Symbol@@"; $mths = @("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec");

ForEach($sLine in $symbolList) { # For each symbol
$ret = GetSymbolInfo $sLine $quotesFolder $minDate; $symbol = $ret[0]; $nextDate = $ret[1]; $symbolMaxDate = $ret[2]; $symbolMaxDateAdj = $ret[3]; $symbolQuoteFile = $ret[4]; $lastQuote = $ret[5];
"Symbol: " + $symbol.PadLeft(12) + ". Next date: $nextDate. Quote file: $symbolQuoteFile" | Out-File $logFile -Encoding OEM -Append;
if ($nextDate -gt $symbolMaxDate) { " We already have data up to maximum configuration date of $symbolMaxDate. Will not request new data for this symbol." | Out-File $logFile -Encoding OEM -Append; continue; }

# ========== Get latest quote from website - Start ==============
@Symbol@@",$symbol);
" Requesting url: " + $url | Out-File $logFile -Encoding OEM -Append; $webpage = ""; $reqRows=0;
$wc = new-object system.net.WebClient; $reqCount++;
try {$webpage = $wc.DownloadData($url); } # Get page from url. This page will contain all quotes for single Symbol
catch { $reqFailed++; " " + $symbol + " - Not Found (web err) `r`n" | Out-File $logFile -Encoding OEM -Append; continue; } # if attempt to get webpage failed go to next symbol

$quotesTxt = [System.Text.Encoding]::ASCII.GetString($webpage); $quotesTxt = $quotesTxt.Replace("???",""); # This variable now contains downloaded quotes

# ---------------------------------------------
# Identifying price
# ---------------------------------------------
#Price is in this string: 174.96
$startStr = "active=`"`" data-reactid=`"29`">"; $endStr = "";
#Price is in this string: data-reactid=`"31`">75.73
$startStr = ""; $endStr = ""; ## <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< This will need to be changed every time Yahoo changes where price is located



if ($quotesTxt.Contains($startStr)) {$quotesTxt = $quotesTxt.Substring($quotesTxt.IndexOf($startStr) + $startStr.Length,500)} # This now includes price and date
else {$reqFailed++; " ** For Symbol: " + $symbol + " new quotes not found (there is no '$startStr' in html that came back)`r`n" | Out-File $logFile -Encoding OEM -Append; continue; } # if that symbol has no quotes, then record errors

if (!$quotesTxt.Contains($endStr)) {$reqFailed++; " ** For Symbol: " + $symbol + " new quotes not found (there is no '$endStr' in html that came back)`r`n" | Out-File $logFile -Encoding OEM -Append; continue; }# if that symbol has no quotes, then record errors
$priceTxt = $quotesTxt.Substring(0,$quotesTxt.IndexOf($endStr)).Replace(",","");
" Found price: $priceTxt" | Out-File $logFile -Encoding OEM -Append;
$reqSucceed++;
# Date: >As of 1:48PM EST. Market open. # or date: >At close: 4:00PM EST # At close: 4:00PM EST

$date = (Get-Date).ToString("yyyy-MM-dd"); # Assuming that always returns todays data. If no, you need to disable intraday requests for those symbols
$dayOfWeek = ([datetime]::ParseExact($date,"yyyy-MM-dd",$null)).DayofWeek.ToString().Substring(0,3)
if ($dayOfWeek -eq "Sat") {$date = (Get-Date).AddDays(-1).ToString("yyyy-MM-dd");}
if ($dayOfWeek -eq "Sun") {$date = (Get-Date).AddDays(-2).ToString("yyyy-MM-dd");}

if($date -ge $nextDate) { # If quote date is after or equal to date we already have
$date + "," + $priceTxt + "," + $symbol | Out-File $quoteIDFile -Encoding OEM -Append; $reqRowsT++;
}
}

$duration = (NEW-TIMESPAN -Start $startTime -End (Get-Date)).TotalSeconds.ToString("#,##0") + " sec.";
(Get-Date).ToString("HH:mm:ss") + " --- Finished. Quotes Requested/Succeed/Failed/Rows: $reqCount/$reqSucceed/$reqFailed/$reqRowsT. Duration: $duration`r`n" | Out-File $logFile -Encoding OEM -append;
$logSummary + ". Quotes Requested/Succeed/Failed/Rows: $reqCount/$reqSucceed/$reqFailed/$reqRowsT. Duration: $duration";

VidasMatelis

When you post your script here, some of the important script details are removed as they are treated as html code. So I cannot confirm if your script is correct.
Also, the value might depend on the country:

Canada: reactid=29
US: reactid=47

trishaan

I am actually in Hong Kong

VidasMatelis

Could you please try following update:

$startStr = "active=`"`" data-reactid=`"29`">"; $endStr = "</fin-streamer>"; 

It appears that in my previous instructions string values were treated as html and hidden.

trishaan

Let me know if this is correct.

Also quick question: Can I use 4 currencies in version 2.4?

VidasMatelis

You would have to change last line in your screenshot with the values I posted. you can copy values from 2 string above.

Really just 3 currencies are supported.

trishaan

Would it be possible for you to email me the text file so I can paste it in my folder?
trishaangill@gmail.com

trishaan

FInally!! It worked thanks