scrubyt’s Activity (RSS)

Was trying out scrubyt for the first time and couldn't get even the simplest of the examples to run. Most were returning some kind of undefined method []=' for ...

In one of my projects I came across that Scrubyt don't support submitting forms without text/select input fields, which is a more common scenario than what one ...

$ cat simple.rb 
require "rubygems"
require "scrubyt"

$ ruby simple.rb 
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_requi...

I checked it with yahoo.com He doesn't seem to fill in the textfield their too.

Could you post the example? It's kind of shooting in the dark without one...

Re: skimr: Are you subscribed to the blog? http://scrubyt.org/blog/ I mentioned it there, though it's a good idea to mention it in the readme.

Due to other resp...

Aha, I didn't know about a new branch. Can you comment on it's fitness for purpose right now? Is it finished enough that people like me can have a go at using i...

Well yeah, this is the worst part of the whole scRUBYt! codebase for sure, very bad initial decisions followed by layers and layers of hacks to fix the problem ...

Thanks for the honesty - as much as the 'wooow scRUBYt! rocks' type of feedback strokes my ego (and who doesn't need some appreciation :-), it doesn't give me m...

I have to be honest - I don't actually understand why the scope of @@original_host is being limited in this way?

Equally I am also not sure why you are using cl...

Cheers, will check it out!

Hi, I built a quick scrapper which returned me some data where the .to_hash looks like an array of hashes, with each hash being just a single named value. In ot...

When scraping pages with urls like: http://somewhere.com/base/mypage

and wanting detail pages for urls like: href="somewhere_else.htm"

Then in mechanize.rb/hand...

Appears that the detail stuff is broken in trunk (and was in the last gem also)

The bug appears to be a scope issue with moving:

def self.get_current_doc_url

 ...

Does it work for you with yahoo.com (without the de?)

Running out for lunch now, but will check it out after I am back!

i now installed it manually and tried to get it working with de.yahoo.com. but i don't even get the script to fill out the textfield.

yahoo_data = Scrubyt::Ext...

hmmm.... wtf it seems it's not yet there. it should be 0.4.1 but for me it installs just 0.4.06... checking it out.

cool, thanks.

but could it be possible that the gem still isn't mirrored at rubyforge?

I think this is what you wanted:

require 'rubygems'
require 'scrubyt'

garbage = <<-SZ
  <html>
    <body>
      <table>
        <tr><td>1</td></tr>
...

e.g. google analytics

Done, phew was not easy...

http://pastie.org/332793

(maybe to wait 5 seconds is too much - try it out and play around with it).

Let me know if it doesn't work (...

Sure, will do!

can you drop me a line as soon as you release it?

OK, done - now I just need to find some time to crank the release out - should happen today, or in the worst case tomorrow!

http://agora.scrubyt.org/forums/...

Solution (yak4):

I modified navigation_actions.rb, firewatir.rb and mechanize.rb. The following is from firewatir.rb.
Exten...

Sometimes we need to wait until the page loading finishes and the DOM is fully parsed - however, an hpricot document is created earlier than this happens in som...

Something like

ruby @@@ fetch :source => :string, '

hey

When using a rake file to load stuff from a screen scrape into a database I ran into a problem 'stack level too deep'. After a lot of scratching around I discov...

Fantastic. Looking forward to it.

Ok, I found the problem... will need to push out one more release ;-)

Yeah, it should (at least that's the idea :-)) however, currently it's throwing

textfield name and //input["@id=p"] is read only. (Watir::Exception::ObjectRe...

If you activate the firewatir agent, it will work!

I know, that's why I didn't got it working. But I thought that Firewatir is included in the new scrubyt and therefore it would be able to scrape the suggestions...

Scrubyt doesn't support javascript, the drop down menu with urls start only by an javascript input, it ask the server and return html, test it with firebug and ...

exactly :-)