StatusEgg 2.0.2

 
 
About
I wrote this app to watch Newegg for when a GTX 680 is in stock. I don't really use this myself, but I though it might help those who're watching for GTX 680 availability. I wrote this because I had most of the code already done in another screen-scraping app. This one just gets the inner HTML text on the Newegg search results page for GTX 680 and processes it. But it will work with a Newegg search for any product, just customize your search options.
 
The application has various ways of alerting you when an item is found to be in stock. Feel free to try it out!
 
License
GNU Lesser General Public License v3.0 (free and open source)
 
Requirements
Microsoft Windows XP, Vista, or 7 (tested in both XP and 7).
Microsoft Internet Explorer 8.0 or later (do your Windows updates)
Microsoft .NET 4.0 Framework (do your Windows updates)

Download
UPDATED: 4/27/2012 
App (v. 2.0.2): http://modarmor.com/docs/StatusEgg.zip
Source code (v. 2.0.2 for Visual Studio 2010): http://modarmor.com/docs/StatusEggSource.zip
 
 
Version History
 
2.0.2

• Bug fix: now updates based on the timer to correctly search by Guided or URL searches instead of always from URL searches.

2.0.1

• Fixed first Column Data config box to show "Name Column Data" instead of "Column Data".

2.0

• New feature: Advanced search configuration. This is a new tab with a ton of options to control nearly every aspect of a Newegg product search. This includes the ability to define what HTML tags, attributes, and values to look for from the DOM when scraping data from a Newegg search page. Also, users are now able to define custom filters for how that data is shown. With this feature, experienced users can correct scraping parameters on their own if Newegg's web site ever changes (future-proofing).
• New feautre: Notification area (tray) status updates. This feature is enabled by default. You can now opt to minimize the application to the Windows notification area. Whenever a product becomes available, your selected sound will play and a notification 'balloon' will appear with information about the product.
• New feature: Saved settings. Users can now save their custom searches along with other application settings.
• Bug fix: addressed the out of memory issue when running many search updates without closing the application.
• When minimized the applications' task bar will now flash when a product is found in stock.
• About tab added.
• Added a couple more options.
• Added the ability to copy the browser output to the Windows clipboard for those wanting to examine the DOM themselves.
• Future-proofing so that changes to the registry settings saving/loading in future versions won't be broken by having existing settings already there from previous versions.

1.1

• Added ability to play sounds when items are in stock. Eight sounds are included, as well as the ability to select a custom sound. Sound looping is also an option. Sound will play every time a search is done and an item is in stock. For example, if you set the search frequency to 30 seconds, a sound will play every 30 seconds if an item is in stock.
• Improved reliability for other Newegg product searches. This will help prevent errors, but at the cost of leaving some fields blank if the application doesn't know how to handle a specific piece of 'scraped' data.

 
Usage
Just extract the StatusEgg.zip file into any folder and run StatusEgg.exe to launch it. It needs the two included DLL files for it to function.
 
If you have any questions or run into any issues, feel free to post here and I'll answer them during workdays.  Enjoy!
 

Ideas/Plans for Future Versions

• New feature to add: single-page in stock updates. Ability to add multiple single pages to check product availability of individual items.
• And separate gain (boost/volume) control for sound playback.
• Add right-click menu items for Windows Vista/7 Aero interface.
• Add ellipses after the truncated product name in the notification area balloon.
• Give an option to change the Buy button column to either just product links or automatically add it to your cart. Buttons could be shown as "View" when not available then "Buy" when in stock. On this note, an option to automatically open a tab in your default browser (Firefox, Chrome, IE, etc.) and add the item to your cart could be enabled. It would have to be disabled by default since we don't accidentally want a ton of items added to your cart.
• Provide an option to write the app's log into a log file instead of just the text box within the app. This might help isolate reasons for app crashes, as well as provided users with a historical log that may be easier to read and parse in the text editor of their choice.

 
Known Issues

• When there is no Buy or Auto Notify image button, the app will not know that it's out of stock. The item is usually out of stock or discontinued in these situations. The "InStock" column will either show "unknown" or be blank if that's the case. This is due to Newegg not having a standard way of displaying these items, since there is no image  on theses where there would normally be a Buy or Auto Notify image. Fortunately the app will never flag these items as in-stock, since they are never in-stock. (Working as intended)
• Sometimes Newegg hides specific search results, depending upon how you performed a search. You might encounter this if you past your own search link into the options tab. StatusEgg will still show these items even though they are hidden in the search results in your browser. When I tested this the items did appear relevant to the search, so I've left this unintended feature in the app. (Will be not changed)
• The app assumes that the data will always exist. If the page includes malformed data or something odd it will error out, but it will not crash the app. (Fixed in 1.1) (Changed in 2.0: since the ability to use custom searches was added, it is possible to ruin your own search, so you can get "Object Reference not set to instance of an object" errors if it can't find the data. In future versions I might change these to more friendly error messages.)
  
• Newegg searches that return "Open Box" items with no Buy or Auto Notify button will also error out. I'll fix this tomorrow. (Fixed in 1.1)

AB Was Here Per Request... 

Downloading!  Let's hope this works the way you say it will!  :D
Although the least you could do is make the app either ding or make it flash in the taskbar when something's in stock, either or, or both.

8IronBob


Downloading!  Let's hope this works the way you say it will!  :D
Although the least you could do is make the app either ding or make it flash in the taskbar when something's in stock, either or, or both.

 
As an alternative, you could use jedi95's application. It beeps. I used it to get 480's when they were released and I can vouch for it working.
 
Here is the original thread
http://www.evga.com/forum...p;m=287409&mpage=1
 
 
Also, nice job on your app cipher_nemo

UPDATE: New version, OP updated.
 

willdearborn
As an alternative, you could use jedi95's application. It beeps. I used it to get 480's when they were released and I can vouch for it working.

Here is the original thread
http://www.evga.com/forum...p;m=287409&mpage=1

Also, nice job on your app cipher_nemo

 
Thanks. I've updated this app to play sounds now.
 

8IronBob
Although the least you could do is make the app either ding or make it flash in the taskbar when something's in stock, either or, or both.

I will do that in the next version: an option to have it minimize to the system tray/notification area and pop up a balloon alert and play sounds when it finds something in-stock.

App throws an .NET error:
 

     See the end of this message for details on invoking 
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Threading.Thread.StartInternal(IPrincipal principal, StackCrawlMark& stackMark)
   at System.Threading.Thread.Start(StackCrawlMark& stackMark)
   at System.Threading.Thread.Start()
   at StatusEgg.frmMain.getGeneratedHTML(String url)
   at StatusEgg.frmMain.scrapeHTMLfromDOM(String strURL)
   at StatusEgg.frmMain.timerUpdates_Tick(Object sender, EventArgs e)
   at System.Windows.Forms.Timer.OnTick(EventArgs e)
   at System.Windows.Forms.Timer.TimerNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
    Assembly Version: 4.0.0.0
    Win32 Version: 
    CodeBase: file:///C:/Windows/Microsoft.NET/Framework/v4.0.30319/mscorlib.dll
----------------------------------------
StatusEgg
    Assembly Version: 1.1.0.0
    Win32 Version: 1.1.0.0
    CodeBase: file:///C:/Users/Micma/Desktop/Egg%20Status/StatusEgg.exe
----------------------------------------
System.Windows.Forms
    Assembly Version: 4.0.0.0
    Win32 Version: 
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System.Drawing
    Assembly Version: 4.0.0.0
    Win32 Version: 
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System
    Assembly Version: 4.0.0.0
    Win32 Version: 
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
Microsoft.mshtml
    Assembly Version: 7.0.3300.0
    Win32 Version: 
    CodeBase: file:///C:/Windows/assembly/GAC/Microsoft.mshtml/7.0.3300.0__b03f5f7f11d50a3a/Microsoft.mshtml.dll
----------------------------------------
System.Xml
    Assembly Version: 4.0.0.0
    Win32 Version: 
    CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
HtmlAgilityPack
    Assembly Version: 1.4.0.0
    Win32 Version: 1.4.0.0
    CodeBase: file:///C:/Users/Micma/Desktop/Egg%20Status/HtmlAgilityPack.DLL
----------------------------------------
Equin.ApplicationFramework.BindingListView
    Assembly Version: 1.2.2446.31718
    Win32 Version: 1.2.0.0
    CodeBase: file:///C:/Users/Micma/Desktop/Egg%20Status/Equin.ApplicationFramework.BindingListView.DLL
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
    <system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.

How to set it to only watch EVGA Products?

Khalus


How to set it to only watch EVGA Products?

Put this line in options:
http://www.newegg.com/Pro...scription=evga+gtx+680

Micma

Khalus


How to set it to only watch EVGA Products?

Put this line in options:
http://www.newegg.com/Pro...scription=evga+gtx+680

EGGcellent....shall be helpful once the 4GB FTW release becomes closer.

You may want to download it again.
I just downloaded it and it works fine, using it to track the Canon 5D Mkiii.

Nice little app Thanks!!!!

Just out of curiosity, Did you write this in C#?

Just an FYI: the app uses the full .NET 4.0. You can't just have the .NET 4.0 Client Profile, you must have the full version.
 

shazam
Just out of curiosity, Did you write this in C#?

Correct, it is written in C#.

Micma
App throws an .NET error:

The app does load an instance of IE in the background (whatever version is installed in your copy of Windows) to scrape HTML from a Newegg search. That IE uses up memory, and unfortunately the compiler for .NET might not do the best job of garbage collection, so the memory usage might grow over time. I'll take a closer look at the app and make sure it's disposing the IE web browser properly every time it does a search. I can either do the garbage collection myself, or just run one IE window (in the background) and keep refreshing that. It loads a separate IE window each search to ensure that both user changes don't and other browser settings don't interfere with the search.
 
If you get this error, you've probably been running it a long time and/or have a low amount of system RAM available. Just close the program and re-launch it after several hundred searches until I can release an update.

You are doing the lords work

matt561
You are doing the lords work

The EVGA Backplate is now available on Newegg, so to refine your search to avoid listing the Backplate, use this url in Options:
 

http://www.newegg.com/Product/ProductList.aspx?Submit=ENE&Description=gtx+680+2gb

 
Also, I'm working on a new version that saves settings and lets you go in and edit the individual HTML scraping values such as the XPath, attributes, attribute values, etc.

I developed a program in Java that automatically checked, filtered, bought and re-sold  Open Box items from Newegg on eBay and Amazon. Worked great for a bout 3 months, made me about $700 in pure profits at the time. 
 
I had to stop development as I had started working 80 hrs a week and I just couldn't have the time to pour on it anymore. But really, the reason why programs of that kind are such a PITA is because the main site, Newegg, changes their markup code every few days. I had grown tired of having to change my finder flags or "tags" whatever you want to call them.
 
I'm hoping one day I can get back to it and maybe port it as a PHP service since .NET is limited to the Windows platform only I'm interested in something that works everywhere (Without using Java lol).
 
Do you program in other environments besides .NET? It'd be nice if we could collaborate on something like that in the future. If you're interested of course.
 
(I hope I didn't stray too far off topic).

Here's a sneak preview of version 2.0 with even more control over your searches. Expect the new version later this week...
 
EDIT: (images removed since version 2.0 is out!)
  
 

shazam
But really, the reason why programs of that kind are such a PITA is because the main site, Newegg, changes their markup code every few days. I had grown tired of having to change my finder flags or "tags" whatever you want to call them.

Indeed. Either you make your code adaptable and do everything in stages that doesn't break the whole process if one thing fails, or you just write a one-off and understand that it might break in the future.
 
 

shazam
Do you program in other environments besides .NET? It'd be nice if we could collaborate on something like that in the future. If you're interested of course.

 
I've done regular C, C++, and Javascript, so I don't mind coding in those languages. I hate the Eclipse IDE, though, as it's so awkward and unintuitive for me. And I hate writing entire Java applications, lol. So yeah, C, C++, C#, and Javascript is all good for me.
 
I also do extensive CSS for my sites. I have dabbled a bit in php, but I really dislike it. I'd rather just code everything in Javascript rather than use php.
 
As for future projects I enjoy writing my own to have full control over it, but I'd be happy to help with someone else's project (ie: advice or troubleshooting for a specific obstacle).

I NEED a new GPU for my new pc, and I don't want to waste money on something else, when what I need is the 680.
 
I have bought the rest of the build already. Its all almost here now, and I still can't get close to a 680. I've been trying to get one every day for what seems like a month now, and I'm losing my mind.
 
 
I can't even buy anywhere else because all I have is a newegg account and bad credit.
 
 
Thanks for the tool. I hope this works.

I hear you on Java and the Eclipse IDE. They are very unintuitive, but at that time I was forced to use Java and Java only as that was the only way for us to make our program cross-platform. I wish I spent all that time doing something else as I've come to really dislike Java. It's very sluggish and the cooked code is not optimized AT ALL.
 
Anyway, I guess you can help me in another way. What would you recommend for me to do to start getting into the VS IDE with C#? I obviously can't do C++ because I suck at C, so there's no point in even going there. But C# is very close to Java, or at least its familiar to me. I'm looking for a book resource and a few reference books to get me started. Would you happen to know of any you could recommend? or maybe a better approach in learning?
 
I have 0 knowledge in C# and the VS IDE. Only used NetBeans, Eclipse and Java. I know a bit of JS, but I mostly do extensive CSS/XHTML work as that's my career field. I'm a Front-End Designer.
 
Thanks man.

Might also be helpful to make an app like this for Amazon watching as well, like when Amazon pops in with pre-orders or even has the actual GPU, which is a rarity, this would be cool.  Although it would have to be true with Amazon when they themselves do something, not when a third-party vendor has them in stock (don't really trust anything other than the legit first-party sellers).

just wanted to say thanks for this! very nice for anyone whos still waiting on one of these bad boys....

cipher_nemo

matt561
You are doing the lords work

Just wanted to give a big thanks, with your app, I nabbed a SC+ on Newegg today...
 
Kudos

This would have been nice when the 680 first came out, I can't remember how long I was at the computer refreshing newegg

Thx for the app. Looking to snag a 680 soon.
Any way to set this to work for the EVGA site so I can try and get a Signature card?
Thx again!

Cool app. One question. Does it check the 680 search result page or each product page? Because by the time the main page updates the card is already gone.

DrSmoke
Thanks for the tool. I hope this works.

No problem, good luck in your hunt! I waited a long time for the 600-series, and love my GTX 680! :)
 
 

KTRe54uWWH9X

Had to sign up and give you props for your newegg price watcher.  This helped me snag the sweet EVGA SuperClocked+ 02G-P4-2684-KR GeForce GTX 680 2GB 256-bit GDDR5 PCI Express 3.0 x16 HDCP Ready SLI Support Video Card as soon as it went on sale.

You might even want to consider putting this on sourceforge/github/codeplex.

props.

Awesome, and congrats! Welcome to the site too.
 
 

Micma
Just wanted to give a big thanks, with your app, I nabbed a SC+ on Newegg today...
 
Kudos

Grats! I love the 680, and I certainly hope NVIDIA can get their act together to ramp up the supply end for their new nm scale.
 
 

shazam
Anyway, I guess you can help me in another way. What would you recommend for me to do to start getting into the VS IDE with C#? I obviously can't do C++ because I suck at C, so there's no point in even going there. But C# is very close to Java, or at least its familiar to me. I'm looking for a book resource and a few reference books to get me started. Would you happen to know of any you could recommend? or maybe a better approach in learning?

Very cool. I love Visual Studio since it pretty much walks you through the stuff you don't want to mess with, then lets you kick it aside and get down and dirty with the stuff you do want to get into. I started with C# from a background in Visual Basic, Lua, and Java. I didn't take any courses or read any books, as I'm much more of a hands-on kinda learner. I just plowed my way through and went from PC/Network tech to getting a job as a Software Engineer. Whenever I run into something I can't figure out, I research it to death and try to understand how other people solved or worked around their issues.
 
The C# language itself is pretty close to both C and Java/Javascript, so if you know either of those it will feel comfortable in a few months of working with it. The language itself is pretty straightforward. The most complicated part is getting to understand the ins and outs of .NET libraries. Those things give me more headaches than all of my language learning combined. So if you want to get into Visual Studio, C#, and .NET , and want a reference, I would pick up a book about .NET instead that uses C# in its examples.
 
 

8IronBob
Might also be helpful to make an app like this for Amazon watching as well, like when Amazon pops in with pre-orders or even has the actual GPU, which is a rarity, this would be cool.  Although it would have to be true with Amazon when they themselves do something, not when a third-party vendor has them in stock (don't really trust anything other than the legit first-party sellers).

I will certainly consider that. I'll take a look at the Amazon web pages and see how they perform their searches. I use Firefox with Firebug to visualize the DOM structure and find ways of isolating the data.
 
 

lightsout
Cool app. One question. Does it check the 680 search result page or each product page? Because by the time the main page updates the card is already gone.

This app checks the search results only, not each product page as that would require even more processing time. But the search results are accurate. The problem is that a lot of these 680s are sold out within a couple minutes of when they become available because they're small quantities (unopened returns, delivery refusals, etc.). We're all waiting for the larger shipments of new stock. On that note, if you're wanting to find the small quantity updates, set your search frequency to 30 seconds or so. Otherwise the default 5 minutes is good for larger quantity shipments.
 
On top of that the app only checks the first page (future versions might check other pages with Javascript injection). So you can set your search results to the maximum of 100 per page to ensure you get all of them. Fortunately there are not too many GTX 680 models out there yet, so the default 20 is usually fine.

I'll certainly be using this app when the FTW model is closer to release.  I was thankfully here the minute the 680s released thus had no trouble getting one of them, but we'll see how it goes with the FTW.