Orange Forum • View topic - GSOC project ideas

GSOC project ideas

General discussions about Orange and with Orange connected things (data mining, machine learning, bioinformatics...).

GSOC project ideas

Postby mailshanx » Tue Mar 06, 2012 14:02

Hello everyone,

I am hoping to participate in the upcoming GSOC programme. I would love to participate through Orange, so i am bouncing some project ideas i have off the Orange dev team/community.

I would like to add some reinforcement learning functionality to Orange. Reinforcement learning techniques are concerned with "learning on the fly", i.e. in situations where an "agent" has choices between various actions it can take and it gets feedback on the results.

Some specific reinforcement learning algorithms i have in mind are:
a) Computation of Gittins Index, which is an optimal solution to the Multi-Armed bandit problem: http://en.wikipedia.org/wiki/Multi-armed_bandit
b) Value iteration/policy iteration: http://en.wikipedia.org/wiki/Markov_dec ... _iteration

So, what do you guys think about this?

regards
shankar.

Re: GSOC project ideas

Postby mailshanx » Sun Mar 11, 2012 6:28

Hello everyone,

Just a gentle reminder: please let me know your thoughts on this proposal! :)

regards
shankar.

Re: GSOC project ideas

Postby marko » Sun Mar 11, 2012 9:27

Dear Shankar,

Orange is a pure data mining tool. For reinforcement learning you also need an agent and an environment that responds to it. How would you integrate that into Orange?

I believe that reinforcement learning is completely orthogonal to what Orange currently does and that it would probably warrant a separate project.

Best regards,
Marko

Re: GSOC project ideas

Postby mailshanx » Sun Mar 11, 2012 15:09

Dear Marko,

I think integration into Orange would require introduction of some additional APIs, the design of which would probably require some discussion before finalizing. However, here is one way in which it can be done:

Let us consider a Gittins index computation for a scenario with 2 arms/choices. Suppose the agent has played each arm a few times, and the result is stored in two lists. We can call gi=orange.Gittins(data) to train our Gittins-index based agent on the existing data.

When the agent needs to play an arm, we can call gi.recommend(), which will spit out either 0 or 1, depending on which arm it recommends.

The result can be appended to the appropriate list, and we call gi.update(data), which updates the learned indices for each arm.

Once we have decided on the online learning API's, we could extend some of the existing classifiers to use in an online setting. One example that comes to my mind is a modified version of regression. We can continuously feed new data to a regression model, that will adjust its weights as the data flows in.

Since Orange already has many implementations of existing classifiers, i think it will be advantageous to start extending online-APIs so that some of those classifiers can be re-used, rather than start a new project from scratch...

regards
shankar.

Re: GSOC project ideas

Postby Mitar » Tue Mar 13, 2012 22:45

Hm, I also think it would be useful to have incremental learners in Orange. Also for Orange canvas, this could prove useful: having a widget which outputs some incremental data and feeds into other widgets.

If you would really like to work on this, please in your proposal propose a good API for the scripting part and also what changes you see that they would be needed for canvas to be able to work on such data.

Re: GSOC project ideas

Postby mailshanx » Sat Mar 17, 2012 17:06

Thanks for your feedback, Mitar. So i take it that incremental learning algorithms are a viable GSoC project? I shall now proceed to post a more detailed GSoC project proposal, to gather further feedback.

Re: GSOC project ideas

Postby Mitar » Sat Mar 17, 2012 17:18

The point is that you have to convince us in your project proposal that it is a viable GSoC project. That you know how to do it, that it is possible to implement it in a given time frame, and so on ... Marko is a bit skeptical, but you can prove him wrong. So prepare your project proposal.


Return to General Discussions



cron