Not supported: Forms without input fields
Reported by Jonas Grimfelt | January 10th, 2009 @ 01:35 PM
In one of my projects I came across that Scrubyt don't support submitting forms without text/select input fields, which is a more common scenario than what one might think.
Example:
Page: zshare.com/download/xxxx
<form action="" method="post" name="form1">
<input id="referer2" type="hidden" value="" name="referer2"/>
<input id="download" type="hidden" value="1" name="download"/>
<input width="219" type="image" height="57" border="0" src="/images/download.gif" name="imageField"/>
</form>
It seems by using instruction such as:
fill_textfield 'user_id', 'foo'
is required to make Scrubyt set a current form, i.e. @@current_form in Scrubyt::Navigation::Mechanize - otherwise it is nil and raises an exception. I consider this as a bug at this point, but quite easy one to solve.
Code that (of course) raise error:
data = Scrubyt::Extractor.define do
fetch url
submit
...
end
Error:
Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/agents/mechanize.rb:71:in
submit': uninitialized class variable @@current_form in
Scrubyt::Navigation::Mechanize (NameError)
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/navigation_actions.rb:58:in `submit'
from zdown.rb:79:in `scrape_download_url'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:75:in `instance_eval'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:75:in `initialize'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:32:in `new'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:32:in `define'
...
I tried to override this behavior with meta programming, but got stuck with some odd errors (maybe because of my short Ruby experience), so instead requesting this as a feature. Something like:
data = Scrubyt::Extractor.define do
fetch url
set_current_form :form1 # ...or just try to use the first found form as default
submit
...
end
Comments and changes to this ticket
-
Peter Szinek January 31st, 2009 @ 01:46 AM
- Assigned user set to Peter Szinek
- State changed from new to open
Makes sense, will look into it for the next rel!
Please Sign in or create a free account to add a new ticket.
With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป
A simple to learn and use, yet powerful web scraping toolkit written in Ruby.