Help betatest SeoTools 4.1 - Google Adwords integration and the SeoTools spider
  • I'm working hard on the next version of SeoTools. Please help my to try out the next release:

    Especially try out the Adwords integration and the Spider!

    http://seotools.nielsbosma.se/releases/SeoTools_4.1.1beta_for_Excel32bit.zip http://seotools.nielsbosma.se/releases/SeoTools_4.1.1beta_for_Excel64bit.zip

    Added in SeoTools 4.1.0beta (2013-03-02):

    • Introducing the SeoTools spider!
    • Take any function in SeoTools and combine into a crawler.
    • Google Adwords integration! Fetch keywords, campaigns, Keyword search volumes and keyword ideas.
    • New functions: BingScraper (check out the wizard), BingOtherDomainsOnIp(), BingLinkFromDomain().
    • Added parameter: Dump({object vector},bool transpose) for transposing results. Added transpose in wizards.
    • Better support for pagination when inserting values in Google Analytics and MajesticSEO wizard. Start index parameter of Google Analytics has been removed. May break your formulas!
    • Added debug metric: ContainsSampledData in Google Analytics. Read more about this.
    • New function WordCount. Returns number of words on a page.
    • Added parameters to IsFoundOnPage(string pageUrl, string countOccurencesOfString, bool ignoreCase, bool textMode, bool distinct) to restrict search in text nodes only and only count distinct links.
    • Fixed problems with Alexa functions.

    In SeoTools 4.1.1beta (2013-04-18):

    • Google PageSpeed integration.
    • Breaking change: BuildHttpDownloaderConfig() renamed to HttpSettings() with modified arguments.
    • Proxy support in HttpSettings:<Proxies> <Proxy Address="87.110.149.80:8080" Username="foo" Password="bar"/> </Proxies> You can enter as many proxies as you want. SeoTools will distribute requests.
    • Breaking change: DownloadUrl() is renamed to GetTextOnUrl()
    • Task pane wizards built for XPathOnUrl, GetTextOnUrl, RegexpOnUrl, JsonPathOnUrl.
    • Changed default response encoding in HTTP settings to UTF-8 as this seems to be the most common encoding on the internet.
    • Dialog for editing HttpSettings() and global http settings (from SeoTools.config.xml).
    • Now (by default) HTTP requests doesn't fail on HTTP codes like 404 and 500. These are now scrapeable. You can set "Fail on HTTP error" in HttpSettings.
    • Fixed bug with certain HTML pages stating to use an encoding that doesn't exists (or is recognized by .NET).
    • Clear cache: =SeoToolsClearCache() Clear all pages from http://nielsbosma.se (use '' as wildcard): =SeoToolsClearCache("http://nielsbosma.se*") From VBA: Application.Run("SeoToolsClearCache","http://nielsbosma.se")
    • Changed HttpStatus() to by default use HTTP request method "GET" instead of "HEAD". This is slower but as pointed out by @lsgbusiness, some servers are configured to return 404 not found. As an optional argument you can choose request method yourself: =HttpStatus("HEAD")
    • Added option to "Collect cookies" between http requests in HttpSettings. Button "Clear cache&cookies" in About resets cookie container.
    • Added "name" option in UrlProperty. =UrlProperty("http://www.nielsbosma.co.uk","name") =>"nielsbosma"
  • 27 Comments sorted by
  • Only 1000 lines were scraped when I tested various functions.

    Win XP (VM) Excel 2007 SE

  • Had anyone reproduced the 1.000 results limit issue where not more than those could be fetched from GA?

  • The new spider looks great Are there usage limits for Majestic queries (standard account)? What are they? Just so i don't blow my monthly allowance in a day..

  • @danos: each url (row in the report) consumes one IndexItemInfoResUnits API unit.

    http://developer-support.majesticseo.com/api/commands/get-index-item-info.shtml

    If you open the MajesticSEO taskpane in SeoTools and login theres a link "MajesticSEO API credits" to check your current credit status.

    /Niels

  • Hi Niels, I'm loving the new Dump with transpose :) and I'm looking forward to trying the AdWords calls.

    Having a couple of problems though - on excel 2010, 64bit with the beta

    A) If I fill in a formula into several (say 10 or 15) cells then I get 'excel is not responding)...smaller 'batches' of cells work okay .

    I'm using Dump (with jsonpathonurl) and an httpdownloader with caching turned on.

    Things that don't seem seem to make a difference: * Caching via (httpdownloader) on/off * Dump vertical/horizontal doesn't

    Things that do seem to make a difference (i.e. stop the crash): * not using dump!

    B) Dump doesn't seem to return a number in the first cell it fills in.... This could be my data but if I have a dump formula in A1, and A1, A2, A3 all contain the value '1' as a result of the dump, SUM(A1:A3) =2 so as a workaround I've been doing SUM(A1:A3)+A1, which gives 3 .....very odd.

    c) Calling from VBA - is there a way to call seotools functions from vba?

    J

  • @Jaspal:

    A) Using dump is only a good idea if you have a single table that you want to "Dump". But if you're getting induvidual cell values like you describe, I wouldn't use Dump. (Dump is a bit of a hack)

    B) I'll look into this.

    C) http://www.blog.methodsinexcel.co.uk/2010/11/16/using-a-excel-dna-function-in-vba/

  • Really interesting new features indeed !

    I checked quickly the Spider feature but it looks like it works only with 1 url. Would it be possible to set a list of urls contained into a text file? (and the spider would be limited to this list of urls)

    This would be a similar feature that is available into the list mode of Screaming Frog.

    Best,

    chris

  • @fassn: it's on my todo! Hopefully in this release!

  • glad to hear it! :)

  • Installed SeoTools_4.1.1beta

    Started getting value errors with XPathOnUrl() in areas that worked with the older version.

    Also the drop down menu for Scraping would not insert any functions.

    Went back to 4.0.7 and these errors went away.

  • Proxy support is great! thanks. Life is some much faster now :D

  • @farsider: can you send me some example XPathOnUrl() queries that isn't working? Note that I've made some changes to Http settings.

    Also the drop down menu for Scraping would not insert any functions.

    They are supposed to show a task pane menu instead of the function wizard. Do you get the menu for Google Analytics?

  • Hi Niels,

    Reinstalled 4.1.1beta and the Google Analytics is visible but nothing happens when I click on it. The same with Google Adword - nothing. Sorry I don't use these, so not much help.

    Clicking on the Scraping Menu - no results when clicking on XPathOnUrl, RegexpFindOnUrl, JsonPathOnUrl or GetTextOnUrl

    I'll send a separate example of the XPathOnUrl() value error

  • Here's one example using 4.1.1beta:

    A2 = "http://www.google.com/search?q=dogs&num=10"

    B2 = "//li[@class="gbt"]/a"

    {=XPathOnUrl(A2,B2)} returns: #VALUE

    Here's the result using 4.0.7 returns:

    Search Images Maps Play YouTube News Gmail Drive More Sign in

  • I think I see why I was getting the #VALUE error.

    In the new Global http settings there was some text in the following:

    Request Headers:

    Header = user_agent Value = SeoTools for Excel %version%

    When I removed these the value errors went away.

    Not sure if this came with the installation?

    The other menu problem above did not resolve.

  • Niels - this is awesome! I have a few observations...

    1) I'm experiencing issues with the 64bit version of the tool - not showing icons and not being able to select some functions - a bit like a previous update. Works fine in 32bit...

    2) Bit of a stupid question... how do you actually know the proxies are working..? Do you think we could get a bit more of a tutorial on setting up proxies?

    Cheers!

  • Excellent plugin.

    I also get nothing When I click on Majestic, Analytics, and Adwords. Is this maybe because they need API keys and such? I got a Developer Token (still waiting for approval), but I did not find where to add it.

    I don't know if this is related or not, but Spider, Majestic, SEOlytics, Analytics, and Adwords have no Icon. !(http://screencast.com/t/2i9TrcML)

  • @RobertoMejia & @mikeyd Make sure you have the latest version of .Net "full"-profile installed. http://seotoolsforexcel.com/links/dotnet

    Also make sure you have propertly unzipped SeoTools.zip to a folder where Excel have write access.

  • I'm also still waiting for developer token approval. It does not seem very easy to get one like they state in their FAQ. Any tips?

    By the way: Really loving the Adwords integration as now I can create a custom report for clients, but would love it get the Adwords data into Excel as well to get it all fully automated.

  • Amazing work, this tool is a lifesaver! I've installed the latest plug-in but also having the same issues as RobertMejia with the missing icons. I installed the latest version of .Net "full" as well. The 4.0.7 version of SeoTools still works for me though.

    If you have any other tips/advice on why it might not work and how to fix it, I'd really appreciate it. Thank you Niels!

  • For some reason I'm unable to get the proxy server feature working. Whenever I provide a proxy server, I either get a timeout or some other error message about a server. If I remove the proxy server, everything works as expected. I'm using a list of public proxy servers from hidemyass-dot-com. Does this only support http proxy servers (as opposed to https and socs4/5)?

    Any help on either a different list to use or how to get this working would be appreciated.

    Thanks much!

  • I'm getting an api warning when attempting to fetch any adwords or keyword data - "you are accessing an adwords api version v201209 that has been discontinued".

    Is anyone else getting this? Is there a fix?

  • I think there's a bug with the Google Analytics integration, if you open an existing sheet, you don't seem to automatically be logged in. If it tries to update while you're not logged in it crashes.

  • Hi guys,

    I'm about to wrap up this version over christmas holidays and would appreciate if you could send me your bug reports for the latest beta so I can fix it.

    Anything else that just has be be in the next version?

    /Niels

  • Hi Neils,

    First of all, great plugin, many thanks for the work you've put into this!

    I am testing the 4.1.1 beta. When I try login in using the Adwords API info, I get a 404 error with the following exceptions. Any suggestions for a fix?

    ************** Exception Text ************** System.Net.WebException: The request failed with HTTP status 404: Not Found. at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall) at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters) at Google.Api.Ads.Common.Lib.AdsSoapClient.MakeApiCall(String methodName, Object[] parameters) at Google.Api.Ads.Common.Lib.AdsSoapClient.Invoke(String methodName, Object[] parameters) at Google.Api.Ads.AdWords.v201209.ManagedCustomerService.get(Selector serviceSelector) at SeoTools.GoogleAdwords.AdwordsManager._LoadAccounts() at SeoTools.GoogleAdwords.AdwordsManager.Login() at SeoTools.UI.TaskPaneWizard.AbstractTaskPaneWizard.Login() at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent) at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks) at System.Windows.Forms.Control.WndProc(Message& m) at System.Windows.Forms.ButtonBase.WndProc(Message& m) at System.Windows.Forms.Button.WndProc(Message& m) at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

  • @Ryan: The Adwords stuff need to be rewritten as they have changed theria apis.

  • Can we expect AdWords API fix soon? Thanks!

Like SeoTools for Excel?

Help me keep SeoTools free!
Please Donate.

Having trouble?

Check Troubleshooting before posting.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Login with Facebook Sign In with OpenID

In this Discussion