POET is an acronym that stands for Partners of Open Education Technologies. This non-profit organization was founded two years ago with the aim of orienting and explaining the benefits of Moodle to those interested in using LMS, as well as helping them integrate it into their existing systems. E-Learn Magazine had the opportunity to talk with Mike Churchward, POET’s Executive Director, who told us more about this group.
E-Learn Magazine: How did the idea of creating the POET Project come up?
Mike Churchward: About two years ago, several Moodle Partners at the time, met about ways to streamline shared processes. One of the more laborious tasks each group was doing, was reviewing and approving Moodle plugins for use by their various installations. So a decision was made to pool resources together and create a Moodle plugin quality program as a separate organization.
ELM: What was the purpose of POET at the time of its founding?
MC: POET’s number one goal was and is, to create an open, publicly facing program that evaluated [Moodle] plugins and provided the results of those evaluations for its members and the community to use. We also wanted to use the same program as a development resource that could be utilized by plugin developers on their own, to help meet quality standards before the plugins were released.
ELM: Who are the members of POET?
MC: The members are: Blackboard, Lambda Solutions and Remote Learner as well as UCLA, Oakland and Minnesota Universities in the US.
ELM: What do you need to do to become a member of POET?
MC: We have not put a lot of effort into increasing the current number of members, but it’s something we want to do; however, at the moment, plugins are our priority. Other companies have approached us and expressed their interest to join, and we are still designing terms for this kind of membership Currently, our members are focussed on contributing resources to plan and develop the plugin quality program, as well as planning the roadmap for the next phases. We will develop a set of guidelines explaining how and what is required to become a member of the POET community, in an open fashion, and invite new members down the road.
ELM: Where do the resources to promote the POET project come from?
MC: Funds and resources come from the community and three of the six founding partners, primarily Blackboard, Lambda Solutions and Remote Learner. We are working on establishing ourselves as a non-profit organization with appropriate funding sources. As a member of the Apereo Incubation program, we are working with Apereo to help define how life as a non-profit will look.
ELM: How is POET related to the Moodle community?
MC: We want to be an additional resource to the Moodle community. Providing our plugin quality program openly to the Moodle community is our first step. We also want to begin co-development of Moodle plugins, and hope to demonstrate that this same kind of development process can contribute to the core Moodle codebase.
ELM: You mentioned POET’s aim of “orienting and explaining the benefits of Moodle to those interested in using LMS.” Does this mean that the plugins you’re testing are available only for Moodle or also for other open-source technologies?
MC: At this time, we’re primarily focused on Moodle. However, our mandate is not strictly Moodle. We want to be able to expand to other open technologies, especially where they integrate with systems like Moodle. Our membership in Apereo can help us there, by connecting us to other open technologies used in education. In the end, it will be our membership that decides what technologies we should embrace.
ELM: Would you say that one of the most important aspects of POET is evaluating plugins that are going to be delivered to a platform to increase its functionality?
MC: At the moment, yes. And we want to be able to help determine what functionality is in demand, and help find the right development to provide that, if it’s not already provided or planned. So, more importantly, we want to be able to help our members get the right functionalities they need for their open educational platforms.
ELM: What do the Plugin Scoring System and Automatic Testing consist of?
MC: Our current plugin quality program consists of a number of parts; some automatic and some manual. We also provide a “score” for various areas such as security and performance, to help gauge the reliability in those areas. The automatic testing process helps validate the plugin to specific Moodle coding and structure standards, as well as identifying areas that should be looked at more closely for security and performance issues. We use the results of these tests as well as our manual reviews to assign scores to various review areas. And we post the results of these tests and reviews, both positive and negative, for anyone to see. This helps plugin consumers to judge whether or not they feel confident using the plugin. Lastly, we provide functional usage reviews to help show how the plugin is used and how well it solves its intended purpose. We have provided several of these initial reviews to the Moodle plugins database already.
ELM: How exactly does the Plugin Scoring System (PSS) work with the Automated Testing (AT)?
MC: The Plugin Scoring System evaluates the testing results of tested plugins using a three-point system. Moodle’s plugin database evaluates plugins with a 10-point score. But we feel the difference between 6 and 9, for example, isn’t very clear and may lead to ambiguities. We prefer to use a simpler system in which 0 is total failure, 1 suggests things that have to be improved, and 2 reflects that tests were perfect. The philosophy behind this scoring system is based on reward; we want to reward those who make an effort to provide and develop the best quality plugins.
But the score isn’t just a number; it’s also accompanied by informative comments that are supplied by POET’s members. We want to provide valuable, collaborative feedback and work with the developers to solve identified problems.
ELM: What kind of plugins have you tested?
MC: We are doing tests for a large variety and series of plugins, including activities, blocks, feedback, authentication and others. Our goal is to test as many plugins as possible so that our processes and workflows become more robust and reliable.
ELM: What kind of technical and security reviews are being carried out by POET with Moodle plugins?
MC: We have based our automated tests on the “moodle-plugin-ci” suite of tests developed in the community, and have added more tests as part of POET. These tests look for code quality, standards adherence, plugin API validity as well as identifying points of concern for security and performance. If the plugin has provided unit tests and Behat tests, the automated process will run them and provide the results. We prefer, and encourage developers to provide unit tests and Behat tests with their plugins and score them higher when they do. We have also done some work to provide unit test and Behat test templates, so that developers who do not have these can start with some and build them out.
We‘re still working on improving and expanding our security and performance testing tools. We’re developing automated code tests to locate and identify known problem areas. At this time, those tests cannot state that a plugin is 100% safe; but they can help identify areas of concern and allow a human reviewer to further inspect and assess.
ELM: What are the plans for POET this year? How does your roadmap look?
MC: One of the foundational objectives is that all members of POET will be able to use the Moodle plugin quality program as their only plugin quality program, by late May. This is a starting point. Ideally, we’d like to integrate our efforts with Moodle’s, so that the Moodle plugin database becomes the single, trusted place for plugin quality.
For our next phases of the plugin program, we have some ideas—primarily in terms of performance. We would like to build large data, high-volume test sites for developers to test their plugins in multiple configurations. There is also a group of Moodle users that are devoted to developing accessibility test suites, and we have talked to them to see how we can help test plugins with respect to accessibility.
ELM: Anything else you want to say?
MC: Our main goal is to enhance and safeguard our members’ experiences. We want to make sure that a requested plugin can demonstrate that its use and purpose are fulfilled, securely and effectively, for all user roles; that is, whether it is used by teachers, students, or administrators. One of our members, the University of Minnesota, have offered its support and assistance, using their user testing laboratories for testing plugin usage and experience.
Thanks to the hard work of POET and its partners, our members will be able to maintain high standards of reliability, flexibility, and responsiveness with their Moodle installations, no matter what plugins they may need.