College Communities Project: Reflection on my own learning

Since April this year I've been working concertedly on planning a project for my MA submission which my employer is supporting me in facilitating.  It's exciting to see six months worth of planning and anxiety coming to fruition.  So I wanted to document some of my early findings.  

In a previous blog entry I shared my literature review and links to my survey tools where I invited peer review.  A link to that can be found by clicking here.

However, since then I have facilitated the first stage of my project, administered my survey and got some early results.  I acknowledge that this early work, in academic terms is likely to qualify as a pilot study (if that).  But the learning process I've been through as a result is what I wanted to document.

Points of Learning:

1.  The Language - I began the project using a survey tool that had been previously devised, tested (Rovai, 2002) and shown to be reliable (Cronbach’s α coefficient of 0.92).  I thought this was a relatively straightforward tool that learners would be able to understand.  So I went ahead as planned and administered the survey to a group of learners.  However, respondents feedback has shown that learner's lack of understanding of what constitutes a community confounds there ability to evaluate whether they are part of one.  In the open ended comments section left at the end of the survey I found quite a few learners responded with, "I wasn't always sure what the question was asking but I tried my best."  

("Words." by ninachildish is licensed under CC BY-ND 2.0)

I'm really keen to use quantitative approaches in this study because I don't see that it's my role to make judgements on other people's experience, but rather to document them.  Does this therefore mean that my research approach should perhaps include an inductive phase whereby learners get some initial training about communities of learning, socialisation, online socialisation and social learning?  Do the learners even recognise that social media and online groups can be forms of community?  

This also manifested in situations whereby learners appeared to provide contradictory statements.  My survey made use of Likert scale kinds of questions which allowed me to translate strength of opinion into quantifiable scores.  However, when these scores provide results such as:

"I felt connected to others on the course" - gained a score of 68%  (itself a very positive indicator) 

...accompanied by... 

"I did not feel a spirit of community" - gaining a score of 66% (a very negative indicator)

... these results seem contradictory.  Clearly either the learners have failed to understand the concept of community as a form of connection, or there is some other issue confounding these results.  Definitely further examination is necessary.  

2.  Power Play on Paper - After providing learners with written assurances as to how their data would be treated confidentially, perhaps naively on my part I thought survey respondents would provide honest responses to question like, "Are you thinking about withdrawing from your course?"  Despite my honest intentions, 100% of learners who responded documented that they were not intending to drop out of their course.  This seemed unlikely and it wasn't until I realised later that learners were frightened of financial repercussions, then I realised why I got the answers that I did.  

("People! Power! People! Power!" by chrisyakimov is licensed under CC BY-NC-ND 2.0)

I suppose that's one of the down sides of relying on self-reporting methods of data collection - the Hawthorne effect manifesting in a digital space.  The learners clearly see me as part of the organisation and therefore a biased participant in my own study.  

At this stage I find myself thinking two things:

(i)  How do I get round that problem?  

(ii)  Fascinating to see the kind of power play I read about in module H819.  It does force me to a position where I question my motivations?  

3.  Interesting ANOVA Results - I adapted the survey tool created by Rovai (2002) and in addition to the 20 statements in the survey I also asked respondents to disclose whether their learning experience had been Face-to-Face, Hybrid or Online.  This enabled me to classify learners into three corresponding groups and compare their mean Classroom Community Scores (CCS) by performing a one way analysis of variance using MS Excel Software.  

The null hypothesis was that there would be no statistically significant difference in the mean classroom community scores between the three groups of learners.  Conversely, my alternative hypothesis was that there is a statistically significant difference in the mean classroom community scores between the three groups.  

(Figure 1:  Screen Shot of Excel ANOVA results).

This implies that there is a statistically significant difference between the CCS Scores reported by learners experiencing different forms of learning and teaching.  Clearly this will require further post-hoc calculations to determine which of the groups has the highest and lowest mean CCS scores.  But, intuitively by examination it seems to suggest that learners who experience Hybrid learning report higher Classroom Community Scores.  With that in mind:

(i)  What are the implications for the organisation?

(ii)  What are the implications for teacher/lecturer education?

(iii)  How reliable are the results given my previous comments?  

  

I can't help at this stage finding myself more than a little bewildered at all of this.  There's such a lot going on with my project and it's only been going a few months.  I don't know if this is any kind of reflection of what life is like as a PhD student, but it seems scary for a MA graduate who is only just contemplating a PhD or EdD Application.  It seems that the more I dig, I'm finding more questions than answers at this stage.  Is this normal? 

I'd love to hear the thoughts and experiences of others.  

You can follow me on twitter @McintoshMclean or contact me here via my blog.  

References 

Rovai, A. P. (2002) Development of an instrument to measure classroom community, The Internet and Higher Education, vol. 5, no. 3, pp. 197 – 211 [Online].  Available at:  https://doi.org/10.1016/S1096-7516(02)00102-1 (Accessed 30th April 2021).