Forum Home Lusternia.com Lore Wiki Vote! Forum Rules VOTE ON TMS!
Home› Life in Lusternia› Common Grounds

Mysteries of the Beauty Seal Revealed

CharitesCharites
in Common Grounds
Hello, everyone.

We are aware that the Beauty judging process is indeed quite mysterious. This has been a deliberate choice - we have felt for these past ten(!) years that it is necessary to maintain this secrecy to protect the sanctity of the judging process. However we recognise that this can create difficulties and, in acknowledging that now, we would like to offer you a brief overview of the judging process so that we can properly address your concerns.

This year, 97 designs were submitted by 47 unique designers. Once the submission deadline had passed, these designs were whittled down to a selection of finalists by us. This process takes a very, very long time. We would estimate that it involves close to fifteen hours when accounting for taking in and acknowledging all of the submissions, whittling designs down, and debating those that we have whittled down (which we do - at great length).

This year there were close to 25 designs that we thought worthy of being finalists - however we cannot put that many before the judges as it would be overwhelming, and so we were forced to narrow the field further. There are a few ways in which we do this that we can outline, as they are useful to bear in mind:

     - We will only put one design per designer through to the finals. This does not mean that submitting multiple designs is a bad thing - it just means that we will pick your best design.
     - We try as much as we can to ensure every commune/city is represented in the finals (this is not always possible but we achieve it most of the time).
     - We try to include a variety of 'styles' of design to offer the judges a broad spectrum of options.
     - We endeavour to ensure that the finalists are a proportional representation of the competition as a whole.

This last point is why you tend to see fewer 'evil' designs; of this year's submissions, perhaps 5-6 were what we would consider dark (note that we do not consider dark to purely be gruesome designs - we hold cakes made of poisonous mercury and socks that depict the Soulless to be equally evil). This is abnormally low. We would love to see more of that genre next year so that the competition can truly represent the breadth of Lusternian designing - but that doesn't mean they are not present! Last year, Krackenor placed fourth with a leviathan-themed book and Narynth placed fifth with a tattoo representing the dark heart of Glomdoring - in fact, a third of the finalists put forth dark or evil themed designs.

Is there a concern that no design along such themes has ever won the Beauty Seal? Perhaps - but that is all the more reason to submit more of them. Either way, we think it does a disservice to those darker designs that have seen great acclaim to suggest that they never do well at all.

Once the field has been narrowed to finalists, normally somewhere between 12 and 15, these designs are then put before the judges. Every single person in Havens, apart from us, votes blindly on the designs. Can we guarantee that they have no idea who designed which? No. Often, designs are submitted that have been made for a God(dess), and so there will always be someone who knows who it is. Others may have been given them as gifts or similar. But as much as we are able to ensure, the voting is done blindly, without designer or cartel or org attached - and in the event that our judges do know the identity of a designer, we trust them to be fair and even-handed.

Each judge ranks the designs in order of their most to least favourite, and points are scored by those designs accordingly. We double check the votes to ensure we have assigned and calculated them correctly. This year, 18 people voted on 12 designs. It was exceptionally close. Between two of the top five there was only a single point; between another two, only seven points. Last year was similarly nail-biting  And many of the votes were radically different, as each of the judges has their own criteria in their heads for what they want in a design (two judges for example ranked this year's winning design as one of their least favourites). As such, we cannot tell you precisely what they are looking for, but here are a few ideas that may help you in the future.

     - The judges are fond of designs that create a clear picture in their heads. If they can easily imagine your design then you have done well.
     - They like designs that have a clear place in Lusternia. Your design does not have to feel like someone walloped them around the head with the Histories, but they want to look at it and know what world it belongs in. This is why 'lore designs' often do very well as the link is immediately clear.
     - They are okay having to look up an unfamiliar word here or there, but if you have made them reach for a dictionary four times in two lines, that is probably not ideal.
     - They do of course like good writing, but nicely formed sentences are in many ways secondary to some of the other requirements. If you have beautiful phrasing but no one can tell what your design looks like, then it will fall down.
     - They like designs that they can read as a single paragraph. Whilst lengthy designs often contain wonderful descriptions, if a design is so long that it becomes difficult to follow then it is probably too long.

We realise this particular point will be controversial, as many Beauty winners and places have been on the longer side. The truth is that these are the shorter ones: we frequently have designs with examined descriptions of 50+ lines submitted for Beauty. The longest design in this year's finals was around half that - not counting Cooking designs, which have more fields and are therefore unique with their length. We would advise that in general you keep your examined descriptions to 25 lines or fewer. Line numbers are judged by the word wrap in the design system, which you can see by looking at any design.

A final disclaimer: over these past 10 years of Beauty competitions we have refined and worked on our judging process considerably. As such everything we have said here is of course subject to change. Back in the beginning, when Ascension was young, judges used to vote the designs out of 100; one year we had close to 25 finalists; as the people within Havens change, so too do the judges' criteria. This explanation represents where we have made it to now, and we will continue to work on making the process as good as we can.

We hope though that this explanation sheds some of the light upon this mysterious process in a way that will be of benefit to all. It should go without saying (but this is the forums, so we will say it anyway) that this opening of dialogue is not an invitation for abuse of ourselves, the judges, or other designers.

You are welcome to ask questions if you have them, though it is very possible that we may not answer them. We must, after all, keep some secrets.

Lovely Charites
Avatar is "Charites" by Arbetta.
«1

Comments

  • EnyalidaEnyalida Nasty Woman, Sockpuppeteer to the Gods
    Part of it is that the Lusternian player design aesthetic, as compared to (for instance) the Aetolian design aesthetic, is dominated by really fantastically over the top and  melodramatic everything. You're either floating around in a cloud of wispy pastel silk constantly or GRIMGRIMDDARKBLOODCORPSES. It's not shocking that a really stylized and sparkly design tends to rise to the top, as they comprise (imo) a significantly larger proportion of total designs. That, and cartel design is meh. 
  • DeichtineDeichtine
    edited February 2017
    Charites said:

    Each judge ranks the designs in order of their most to least favourite, and points are scored by those designs accordingly. We double check the votes to ensure we have assigned and calculated them correctly. This year, 18 people voted on 12 designs. It was exceptionally close. Between two of the top five there was only a single point; between another two, only seven points. Last year was similarly nail-biting  And many of the votes were radically different, as each of the judges has their own criteria in their heads for what they want in a design (two judges for example ranked this year's winning design as one of their least favourites). As such, we cannot tell you precisely what they are looking for, but here are a few ideas that may help you in the future.

    This could be something to clarify then. 


    There's some fairly good judging guides and well judging itself is a fairly high skill level thing, often professional judges do take courses and there is clear set reasons and criteria for the point scoring system. If your getting such varied results between judges then the judges themselves may need to be given more clear criteria and training on what is required to get high scores.

    You could set point levels and build from there.

    Like 0-3 points is for  Skill and technique. Pure writing style and quality. 0-3 points for uniqueness and creativity. Etc etc.

    EDIT: Like you can look at judges scoring baking or art contests and the scores rarely vary because the judges know and agree before hand what the criteria is for scoring, taste, look etc etc.
  • DeichtineDeichtine
    Hmm good point trem, feed back would be super good to have. 

    Your pie was too dry etc. Like other judging competitions.
  • TamsinTamsin
    I'd probably be far less frustrated to even have acknowledgement that somebody up there liked it. I'm quite confident in my design skills usually, but every single time I have entered Beauty (which has been more times than I have actually bothered to count until the last few years) I have come out of it extraordinarily frustrated. My hormonal pregnant self's reaction yesterday was so far not pretty that I couldn't even be grateful a friend placed, because I've tried long, I've tried short, I've tried dark and light, different trades, and apparently I'm just way more amateur than I think I am. It's very hard to take Beauty objectively, I find.
  • TamsinTamsin
    Sylandra said:
    Tremula said:
    One thing I've always wished would happen is that our designs could come back to us with even one comment from the judges (who remain anonymous, of course) so that we don't get discouraged and keep going. I know that just about every year I've played I've submitted to Beauty, and every year I try to come up with something better to try and place or win THIS time. But since I never know what was liked about my designs or what wasn't, I have no idea in how to make them better except to my own personal aesthetic. 
    I get what you're saying. But as someone who has reviewed slush piles before, that can get really time consuming, really fast.

    Let's play with this for a moment. According to the Charites, that means 97 designs would have to get a personalized response. 97! And the answer could be as simple as, "We liked this but we liked some of the other designs more that were doing a similar thing." Even if each designer just got one response, that's still 40+ responses to write up. And now that we better understand why the actual judging is long and time consuming, that means we'd have to wait even longer for a Beauty result (or demand a quicker deadline for Beauty) to make time for writing those responses.

    I don't really think that's practical.

    I volunteer as a fiction reader for a literary journal and we sometimes get 300+ stories to respond to. We have a canned rejection letter we give most of the time, because personalized feedback is more time spent on an individual submission. Time is a limited commodity, and no matter how much time you spend explaining why you said no, in the end the answer is still a no. Think of the Beauty contest "honorary mention" as a personalized rejection: you were very close, but in the end, others beat you out.

    And remember: the no in Beauty, like in judging any creative work, is a subjective no. Like the Charites say, there is turnover in the Havens. Maybe what you wrote this year would have gone better with judges from two years ago. There's really no way to know. You just have to be confident in your own work you put forward, and hope someone else sees what you're trying to do.
    Keep in mind that the Charites and reviewers are already doing something similar from an objective structural standpoint for weeks at a time beforehand. It is unlikely to be nearly as much work as we can only imagine all the reviewing is, which takes a MUCH more specific eye to detail and flow.
  • TremulaTremula Banished Quasiroyal
    Right, @Sylandra, I understand that, but I'd be happy if this letter came out even two weeks later. I'm not asking for it to happen right after Beauty ends, I agree, that'd be chaos. I can barely handle 26~ people sending me mafia night messages and getting personalised responses. However, if the judges are already writing down comments (which most judges do in all the competitions I've seen/participated in throughout the years), matching a few similar notes and sending them out over the course of a few weeks, maybe even a month, would still be a lovely thing to receive.
                          * * * ＷＲＡＣＫ ＡＮＤ ＲＯＬＬ ＡＮＤ ＤＥＡＴＨ ＡＮＤ ＰＡＩＮ * * *
                                         * * * ＬＥＴ'Ｓ ＦＥＥＬ ＴＨＥ ＦＥＡＲ ＯＦ ＤＥＡＴＨ ＡＧＡＩＮ * * *
              * * * ＷＥ'ＬＬ ＫＩＬＬ ＡＮＤ ＳＬＡＵＧＨＴＥＲ, ＥＡＴ ＴＨＥ ＳＬＡＩＮ * * *
      * * * ＩＮ ＲＡＶＡＧＩＮＧ ＷＥ'ＬＬ ＥＮＴＥＲＴＡＩＮ * * *

    Ixion tells you, "// I don't think anyone else had a clue, amazing form."
  • SylandraSylandra Join Queue for Mafia Games The Last Mafia Game
    edited February 2017
    I think it's still a lot of work to explain to 40+ people why you chose not to award them a win for their design. 

    Also, it's a giant can of worms. What if you don't like the feedback you get?
    Daraius said:
    "Oh yeah, you're a naughty mayor, aren't you? Misfile that Form MA631-D. Comptroller Shevat's got a nice gemstone disc for you, but yer gonna have to beg for it."
  • FalaeronFalaeron Jolteon
    edited February 2017
    I don't think it's fair to compare the standard reviewing process with judging for a competition like Beauty.

    You are right, for the most part reviewing is an objective process. Do the designs contain any spelling or grammatical errors? Do they comply with the commodity requirements? Do they fit within the physical bounds that crafting items should fit within as determined by the Charites? All of these are objective. It doesn't matter if the reviewer doesn't like that one of the sentences is boring like "This is a simple box." For all intents and purposes, this is a valid sentence to use within the design system and if the rest of the design is also valid then this design should be approved.

    Beauty judging, like any judging for writing competitions, is a subjective process. There is no set of criteria where if you can tick all of the boxes you are guaranteed to win, place or even get recognition for it. Who is to say that "This is a simple box made of wood." is a better sentence than "This is a simple wooden box."? You might like one more than the other and I might like the other one more. If it was just for reviewing for design approval purposes both are valid, but since the judges are being asked which is better it is entirely subjective.
  • OtheroOthero
    Tamsin said:
    I'd probably be far less frustrated to even have acknowledgement that somebody up there liked it. I'm quite confident in my design skills usually, but every single time I have entered Beauty (which has been more times than I have actually bothered to count until the last few years) I have come out of it extraordinarily frustrated. My hormonal pregnant self's reaction yesterday was so far not pretty that I couldn't even be grateful a friend placed, because I've tried long, I've tried short, I've tried dark and light, different trades, and apparently I'm just way more amateur than I think I am. It's very hard to take Beauty objectively, I find.
    TBH, that's your problem and not the Charities. They have no need to pat you on the back or acknowledge if they did or did not like your design. If you want feedback ask other designers. And in a subjective contest like this I can tell you from experience that the feedback won't be useful. If you don't have glaring flaws or errors the feedback will end up being vague and subjective. 

    Work this out with a friend, take your designs and take the winning designs and be critical and find out what you are missing. Because trying short, long, different trades, light, and dark is not enough. There is no magic combination that wins. You need to critically work over your designs and find that je ne sais quoi that you are missing.
  • DylaraDylara
    I said what I wanted to say in the other thread, but thank you, Charites, for clarifying the process behind the Beauty submission. Hopefully next year will be just as fun and maybe even we'll see a shake up in how the contest is done/winner circle. :)

    Avurekhos says, "Dylara's a PvP menace in my eyes, totes rekting face."

    The eye of Dylara materialises in your hands and flings itself around your neck, tightening incomprehensibly until it is irremovable.
    Perfectly clean, this eyeball has been wrenched from the socket of Dylara. It has been animated by some unusual force, constantly looking around itself as if in shock or fear. It is bathed in a light covering of white flames that roll endlessly over its surface. A single chain of empyreal metal pierces either side of the eye, allowing it to be worn around the neck.


  • TamsinTamsin
    Othero said:
    Tamsin said:
    I'd probably be far less frustrated to even have acknowledgement that somebody up there liked it. I'm quite confident in my design skills usually, but every single time I have entered Beauty (which has been more times than I have actually bothered to count until the last few years) I have come out of it extraordinarily frustrated. My hormonal pregnant self's reaction yesterday was so far not pretty that I couldn't even be grateful a friend placed, because I've tried long, I've tried short, I've tried dark and light, different trades, and apparently I'm just way more amateur than I think I am. It's very hard to take Beauty objectively, I find.
    TBH, that's your problem and not the Charities. They have no need to pat you on the back or acknowledge if they did or did not like your design. If you want feedback ask other designers. And in a subjective contest like this I can tell you from experience that the feedback won't be useful. If you don't have glaring flaws or errors the feedback will end up being vague and subjective. 

    Work this out with a friend, take your designs and take the winning designs and be critical and find out what you are missing. Because trying short, long, different trades, light, and dark is not enough. There is no magic combination that wins. You need to critically work over your designs and find that je ne sais quoi that you are missing.
    You are assuming I don't do all these things because I didn't think it wise to write a wall-of-text and hijack - I do, and in fact, two of those who placed this year saw my design before it was ever submitted. I am actually quite thorough in attempting different things and getting input from other people. 

    Also, nobody was asking for in-depth critiques from the judges, or at least that wasn't the impression I got out of it. But obviously, the idea of some kind of acknowledgement that hey, this one person liked your concept, is outlandishly selfish and childish. My bad.
  • OtheroOthero
    Sorry for being harsh but I'm glad you realized that. It's selfish to ask someone to make 97 unique attaboys on top of the 15 hours of volunteer work they already do. Thanks for being understanding.
  • LuceLuce Fox Populi
    Taking this from a different direction, would it be possible for, after Beauty is announced, a system generated message to kick back which of your designs was given final submission, and what the highest vote it got was? No personal message needed, just a clear. "Of your submissions, Cookie 1111 'a heart-shaped royal tart' was submitted for judgement. It was rated up to a 7 on a 10 point scale."
  • FalaeronFalaeron Jolteon
    Luce said:
    Taking this from a different direction, would it be possible for, after Beauty is announced, a system generated message to kick back which of your designs was given final submission, and what the highest vote it got was? No personal message needed, just a clear. "Of your submissions, Cookie 1111 'a heart-shaped royal tart' was submitted for judgement. It was rated up to a 7 on a 10 point scale."
    I really doubt any of the judging is done through any mechanical system built into the game so "system generated message" would really still mean more volunteer work in the end. 
  • DeichtineDeichtine
    Falmiis said:
    I don't think it's fair to compare the standard reviewing process with judging for a competition like Beauty.

    You are right, for the most part reviewing is an objective process. Do the designs contain any spelling or grammatical errors? Do they comply with the commodity requirements? Do they fit within the physical bounds that crafting items should fit within as determined by the Charites? All of these are objective. It doesn't matter if the reviewer doesn't like that one of the sentences is boring like "This is a simple box." For all intents and purposes, this is a valid sentence to use within the design system and if the rest of the design is also valid then this design should be approved.

    Beauty judging, like any judging for writing competitions, is a subjective process. There is no set of criteria where if you can tick all of the boxes you are guaranteed to win, place or even get recognition for it. Who is to say that "This is a simple box made of wood." is a better sentence than "This is a simple wooden box."? You might like one more than the other and I might like the other one more. If it was just for reviewing for design approval purposes both are valid, but since the judges are being asked which is better it is entirely subjective.

    Well professional writing and art judging contests have these things already and both are fairly subjective. 

    Its fairly common for the judges in competitions to have very similar scores because the criteria of the contest is set out clearly before hand for the judges and all the judges are essentially singing from the same hymn sheet. If you have a case of many judges are giving vastly different scores then the contest is left to be a very subjective thing but well we can improve that fairly easily by implementing some standard judging practices. 
  • AlaksanteriAlaksanteri
    edited January 2019
    -
  • BreandrynBreandryn
    I do not think anyone is saying that the admin are unfit to judge the contest, and I am sorry that is the impression our feedback gave you. Rather, I think what we are saying and feeling is that the design goals are nebulous, which the Charites and yourself have confirmed: there is no standard for judging that we can base our designs around and use as guidance, which leads to a frustrating feeling of not being able to find an ineffable magic formula to appease the sundry disparate judging views.
  • MaligornMaligorn Windborne
    It seems like the highest placing Beauty competitors come from Hallifax and Glomdoring. It's really not the worst thing in the world to take some hints from them.

    For me, it seems like Beauty is down to this:

    1). Perfect syntax and interesting writing style; absolutely clear but reasonably intricate.

    2a). Thematic to the Basin, but better if not heavily affiliated with a particular org.
    2b). Particularly good if it's relevant to the Ascension event or any event in the past year.

    3). Not too long, not too short. Similarly, not too over-described, not too under-described.

    4). Unique, fresh idea without being totally over the top.

    image
  • SylandraSylandra Join Queue for Mafia Games The Last Mafia Game
    Breandryn said:
    I do not think anyone is saying that the admin are unfit to judge the contest, and I am sorry that is the impression our feedback gave you. Rather, I think what we are saying and feeling is that the design goals are nebulous, which the Charites and yourself have confirmed: there is no standard for judging that we can base our designs around and use as guidance, which leads to a frustrating feeling of not being able to find an ineffable magic formula to appease the sundry disparate judging views.
    I dunno, I think the bullet points in the OP of this thread are actually really useful in that regard, personally.

    (Also this pro-tip is one I have found useful elsewhere beyond Lusternia: if you want to win any kind of creative writing competition, look at the examples of people who have done well before you. Make your own list of what they do and what they have in common, then assess your own work with those points in mind. Can confirm success with this method both in Lusternia and out.)
    Daraius said:
    "Oh yeah, you're a naughty mayor, aren't you? Misfile that Form MA631-D. Comptroller Shevat's got a nice gemstone disc for you, but yer gonna have to beg for it."
  • BreandrynBreandryn
    Sylandra said:
    Breandryn said:
    I do not think anyone is saying that the admin are unfit to judge the contest, and I am sorry that is the impression our feedback gave you. Rather, I think what we are saying and feeling is that the design goals are nebulous, which the Charites and yourself have confirmed: there is no standard for judging that we can base our designs around and use as guidance, which leads to a frustrating feeling of not being able to find an ineffable magic formula to appease the sundry disparate judging views.
    I dunno, I think the bullet points in the OP of this thread are actually really useful in that regard, personally.

    (Also this pro-tip is one I have found useful elsewhere beyond Lusternia: if you want to win any kind of creative writing competition, look at the examples of people who have done well before you. Make your own list of what they do and what they have in common, then assess your own work with those points in mind. Can confirm success with this method both in Lusternia and out.)
    Maybe. I'm not particularly invested myself - I've never won a crafting contest in the 10+ years I've played IRE, and I don't really expect to. I'm more of a fiction writer and my style tends towards the simplistic. I was simply trying to give some feedback based on what I've been hearing. The Charites said that the votes were radically different, which confirms that each judge has a unique scale for judging.
  • OrventaOrventa
    Maybe some year there could be an experiment of sorts where we write to an esoteric theme prompt or phrase or vision when the seals break. (If you're not present for the initial breaking you could touch the beauty seal to get the vision.) Then we write our interpretation of what that vision meant to us. Sure, it would mean less time planning in advance, but then we'd get to show off our quick thinking and inventiveness. Creative writing speed prompt, like @Breandryn 's Iron Chef style cooking contest.


    I'm a consent-based roleplayer! Kindly ask first, and I will return the favour. Open to developing tinyplots.
    Atlantis is my client of choice! (Guide)
  • BreandrynBreandryn
    I totally jacked the Iron Chef contest from Aetolia. Every 2-3 months they hold a much more polished version of that with fun guest NPC judges and a nice prog for creating the items on the spot.
  • DylaraDylara
    Nienla was telling me about the Iron Chef contest, and it sounds amazing. One of the things from Aetolia I wish Lusternia had for sure.

    Avurekhos says, "Dylara's a PvP menace in my eyes, totes rekting face."

    The eye of Dylara materialises in your hands and flings itself around your neck, tightening incomprehensibly until it is irremovable.
    Perfectly clean, this eyeball has been wrenched from the socket of Dylara. It has been animated by some unusual force, constantly looking around itself as if in shock or fear. It is bathed in a light covering of white flames that roll endlessly over its surface. A single chain of empyreal metal pierces either side of the eye, allowing it to be worn around the neck.


Sign In or Register to comment.