Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

Not supported: Forms without input fields

#13

In one of my projects I came across that Scrubyt don't support submitting forms without text/select input fields, which is a more common scenario than what one might think.

Example:

Page: zshare.com/download/xxxx


<form action="" method="post" name="form1">
  <input id="referer2" type="hidden" value="" name="referer2"/>
  <input id="download" type="hidden" value="1" name="download"/>
  <input width="219" type="image" height="57" border="0" src="/images/download.gif" name="imageField"/>
</form>

It seems by using instruction such as:


fill_textfield 'user_id', 'foo'

is required to make Scrubyt set a current form, i.e. @@current_form in Scrubyt::Navigation::Mechanize - otherwise it is nil and raises an exception. I consider this as a bug at this point, but quite easy one to solve.

Code that (of course) raise error:


data = Scrubyt::Extractor.define do
  fetch url
  submit

  ...
end

Error:

Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/agents/mechanize.rb:71:in submit': uninitialized class variable @@current_form in Scrubyt::Navigation::Mechanize (NameError)

from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/navigation_actions.rb:58:in `submit'
from zdown.rb:79:in `scrape_download_url'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:75:in `instance_eval'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:75:in `initialize'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:32:in `new'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:32:in `define'
...

I tried to override this behavior with meta programming, but got stuck with some odd errors (maybe because of my short Ruby experience), so instead requesting this as a feature. Something like:


data = Scrubyt::Extractor.define do
  fetch url
  set_current_form :form1 # ...or just try to use the first found form as default 
  submit

  ...
end

Reported by Jonas Grimfelt · January 10th, 2009 @ 01:35 PM

State: open
Milestone: none
Assigned to: Peter Szinek Peter Szinek

Activity

  1. Peter Szinek
    Peter Szinek
    • Assigned user set to Peter Szinek
    • State changed from new to open

    Makes sense, will look into it for the next rel!

    January 31st, 2009 @ 01:46 AM

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.