Confused JVM? Kill it!

We have a customer where we’ve started to get memory problems in Alfresco recently. Those kind of problems can be very hard to pinpoint but for this particular client we’re almost sure what’s causing the memory problems. Unfortunately for us, this knowledge doesn’t prevent the memory problem. The bad thing with the JVM (in this case) is that even though a memory problem has occured, the JVM is left in a running state, although it’s not in a good running state… This particular Alfresco solution is clustered, and the memory problems ejects the server from Alfresco’s Hazelcast cluster but the LoadBalancer cluster still thinks the server is in the cluster which leads to a lot of problems down the road :(

Our customer have a very good organization around Alfresco, and if the JVM in which Alfresco lives should die when a memory problem occurs, it can be restarted in a breeze. In order to achieve this there is a JVM parameter (-XX:OnOutOfMemoryError) which can be used to execute a script when such an error occurs.

Below is how we solved this for our customer, a step-by-step instruction how to achive this. The server OS is Ubuntu 12.04.

  1. Install the package mailutils if not already installed.

    sudo apt-get install mailutils
  2. Create a shell script somewhere in your installation path.

    nano -w /opt/alfresco/current_version/bin/mail-and-kill.sh
  3. Paste this content into the script.

    #!/bin/bash
    
    # First argument  : process id
    # Second argument : server port
    # Third argument  : module (repo, solr or share)
    
    PROCESSID="$1"
    PORT="$2"
    MODULE="$3"
    FROM="noreply@example.com"
    TO="it-operations@example.com"
    SUBJECT="Tomcat shut down on $HOSTNAME:$PORT ($MODULE)"
    FILENAME="/tmp/mail-and-kill.txt"
    
    rm -f $FILENAME
    
    echo "Server  : $HOSTNAME" >> $FILENAME
    echo "Port    : $PORT" >> $FILENAME
    echo "Module  : $MODULE" >> $FILENAME
    echo "Message : Server got an java.lang.OutOfMemoryError and java process is killed" >> $FILENAME
    
    mail -a "From: $FROM" -s "$SUBJECT" $TO < $FILENAME
    
    kill -9 $PROCESSID
        

    The script takes three parameters. Process id (jvm process), port of Tomcat (HTTP port), and the module which caused the problem (repo, solr or share).

    In order for this to work, a local mail server has to be installed. For our client we’ve installed postfix which acts as a mail relay server.

  4. Add the following to the Tomcat startup parameters (for example setenv.sh).

        JAVA_OPTS="$JAVA_OPTS -XX:OnOutOfMemoryError='/opt/alfresco/current_version/bin/mail-and-kill.sh %p 8080 repo'"
        
  5. Restart Alfresco and force some nasty code to kill it :) and watch how you get a mail and the JVM is killed.
This entry was posted in Uncategorized. Bookmark the permalink.

52 Responses to Confused JVM? Kill it!

  1. Jim says:

    Wow, this paгagrɑph is pleasant, mmy sister is analyzing such things, sso I am going to inform her.

  2. Le régulateur électronique BOSCH offre une meilleure adaptation au matériau elegance la molette de choice de la cadence
    d’oscillation.

  3. Attention, les guides de coupe (sabots) de
    la tondeuse Babyliss E709E sont fragiles et se cassent tres rapidement (Commentaires clients Amazon).

  4. Desta maneira, evite viver universalidade tão a organizado entretanto se apoquentar graças a primordial, mostre que vassuncê sabe que
    primordial passa, que a estímulo porém igual duração bem
    como que momentos agradáveis contudo desagradáveis fazem pedaço da início do que algum todo.

  5. Sou fundado em cima de dentistaria no entanto a despeito de a negativo decidir insignificância com
    blogs, programação, webdesign, mercadologia do dedo
    ou…ou não existente pena condição, criei este
    sítio eletrônico sem ajuda, pesquisando nisso livros,
    e-books, video-aulas nem integralmente este essência que achamos na internet.

  6. I’m really loving the theme/design of your blog. Do you ever run into any web browser compatibility issues?
    A couple of my blog visitors have complained about my website not working correctly in Explorer
    but looks great in Opera. Do you have any recommendations to help fix this issue?

  7. best deals says:

    HTC has earned very much popularity in the mobile market for their Windows
    based Smartphones. It’s amazing how having something dry on your feet wkll
    change your whole feeling on a soggy day. The mobile phones deals are available with various leading network providers
    such as Virgin, Vodafone, O2, T-mobile and Orange.

  8. Thanks to their trade contacts, travel professionals will
    help you negotiate large-group discounts and arrange your entire itinerary particulars so your family reunion or
    your company retreat goes off with out a hitch.

  9. Darrel says:

    Sergio luiz conserto de televisão Niteroi (Rio de Janeiro, Rio de
    Janeiro) Conserto de televisão de todas as
    marcas.

  10. Essa Senotec desentupidora em Abertura conta com técnicos intensamente qualificados em lidar com nenhum gênero de
    gênero de de obturação, pois além da experiência na solução desde todos e cada um dos complicações com
    encanamentos e também com escoamento de esgoto, conhecem essa localidade uma
    vez que nenhuma outra desentupidora em Buraco.

  11. Cela va déterminer la taille de votre plancha et c’est pour
    nous très important.

  12. Weeds usually are not a lot totally different from the
    plants by way of their necessities for survival.

  13. This article offers clear idea in support of
    the new users of blogging, that genuinely how to do running a blog.

  14. There are lots of DIY ideas which you could use on those colorless cabinets in your
    kitchen to bring back that sparkle. Therefore, white kitchen cabinetry has been always a timeless option.
    With its classic taste, “Melograno” is an italian kitchen that is designed to get a great success in America, a market where Traditional taste its
    always on and Transitional is becoming day by day a new exclusive trend.

  15. Thanks for sharing your thoughts about kursus bahasa inggris jakarta barat.
    Regards

  16. kreg jig k4h says:

    Wohh just what I was looking for, appreciate it for putting up.

  17. It might even become somewhat of a weight if you contemplate it.
    Selamat menikmati informasi yang kami sajikan dan semoga informasi ini
    dapat membantu anda dalam bermain judi online dan taruhan bola.
    With the president announcing his support for construction of
    the southern leg of the KXL, even offering to “cut through the red tape” to speed
    up the process, this would seem an opportune time to put him
    on the spot with our “reasonable compromise” as embodied by the Terry bill.

  18. see says:

    This web site definitely has all of the information I needed concerning this subject and didn’t know who to ask.

  19. psimundi.com says:

    Thank you for another excellent article. Where else may
    just anybody get that kind of information in such a perfect method of writing?

    I’ve a presentation subsequent week, and I’m at the look
    for such info.

  20. The Symbol Turbine uses the SQL database of numerous sites and manipulates the amount of free tokens
    required from the individual of Chaturbate expression hack application to fairly share in personis
    dialect.

  21. curry 3 says:

    She just lifts up the tongue and slips them on however they nonetheless seem like large girl sneakers!

  22. uggs on sale says:

    The Cheap UGG Boots Sale UK is the unfailing model of
    the the very first starting,the UGG Boots Sale UK producers just bought two or three syles of UGG boots.Amongst them,there had UGG Basic Tall.

  23. Thankfulness to my father who told me on the topic of this blog, this blog is in fact amazing.

  24. I’ve been gaming for 25 years, and the price for games have
    stayed the same, but the cost to make games has gone up many times over.

  25. This Big Fat Lotteries teams all vary in size and will
    depend on the lotto game played, you will also find along
    with the introduction of a Superenalotto syndicate competitors of
    100 members. A travel agent who specializes in Disney Destinations will know what discounts are available and will help you
    take advantage of the best deal for your family. With the president announcing his support for construction of the southern leg of
    the KXL, even offering to “cut through the red tape” to speed up the process, this would seem an opportune time
    to put him on the spot with our “reasonable compromise” as embodied by the Terry bill.

  26. Data togel says:

    “If you want to do a quick tour of the four main Disney Theme Parks, I’d recommend
    5 days; one for each Theme Park plus an extra day to go back to either
    Epcot or the Magic Kingdom, both of which are difficult to do in only one
    day. Don’t’ forget to click on the Slide Show and
    Video to your left. One more thing that you need to give thought to is working with all of the information that toto singapore has to provide to you.

  27. Wilda says:

    There are a few intriguing points in time in this post but
    I do nnot determine if I see these users center to heart.

    There is some validity but I will take hold opinion until I investigate it further.
    Excellent article , thanks and we want more! Added onto FeedBurner
    too

  28. Nidia says:

    Atendimento ao cliente tradicional deixa a desejar, com nota 2.24 no Reclame
    Aqui, 100% das reclamações atendidas e apenas 29.5% solucionadas.

  29. Blog yang bagus btw gua berrbagi berdita untuk jadi dewa poker cekidot
    pokertogelmania.
    Info cara bermain kattu hingga prediksi togel. Barangkali jujga sahabat ada yang butuh
    referensi situs game uang aslii terbaik ɗi Indonesia kunjungi pokertogelmania.

  30. The driver of a vehicle approaching a totally or partially blind or otherwise physically disabled pedestrian who is
    using a service dog, shall take all necessary precautions to avoid injury; any driver
    who fails to take such precautions is liable in damages for any injury caused the pedestrian.

  31. The virtual world has received error messages within browsers prior to now,
    which have included gamers questioning the best way to
    restart their internet browser for Roblox after they can not entry the login web page.

  32. In this manner, the roblox gaming website is providing the exclusive companies for the
    players to get the attention-grabbing game play.

  33. Whats up are using WordPress for your site platform?
    I’m new to the blog world but I’m trying to get started and set up my own.
    Do you need any html coding knowledge to make your
    own blog? Any help would be greatly appreciated!

  34. Dentist Near says:

    The Fix: Your dentist can offer you a night guard, a small
    mouth piece that will safeguard your teeth while you sleep– and protect your investment in the dentistry that’s already there, Jones states.

    The process of sedation dentistry helps attending to these problems by decreasing the client’s
    apprehensions.

  35. Furthermore, a enterprise should additionally
    function inside the demarcation of guidelines and insurance policies as formulated sometimes by the federal government of
    state or nation.

  36. Personally, I have all the time steered away from desperation, assuming that the shows get extra of those letters than something.

  37. let windows says:

    I dugg some of you post as I cogitated they were handy
    very helpful.

  38. kd 9 says:

    The Nike Zoom Hyperfuse basketball shoe, for instance, uses a
    mix of synthetic leather, mesh and TPU film to maintain your toes cool.

  39. apple says:

    Hi there, this weekend is nice in support of me, for
    the reason that this time i am reading this great informative paragraph here at my house.

  40. Downloading Facetime for PC isn’t possible as a result
    of no official model of facetime windows PC has been launched yet.

  41. It’s actually very complicated in this busy
    life to listen news on TV, so I only use world wide web for that purpose, and get the latest information.

  42. togelplus says:

    For newest information you have to visit web and on the
    web I found this web site as a finest site for newest updates.

  43. Attractive section of content. I just stumbled upon your weblog and in accession capital to
    assert that I acquire actually enjoyed account your blog posts.

    Anyway I will be subscribing to your feeds and even I achievement you access consistently fast.

  44. Hi! This is my first visit too your blog! We are
    a group of volunteers and starting a new project in a community in the same niche.
    Your blog provided us useful information to wor on. You have done a outstanding job!

    Feel free to visiit my blog post; 1K Daily Profit

  45. Yes! Finally something about top news.

  46. steam keys says:

    This piece of writing will help the internet viewers for setting up new weblog or even a weblog from
    start to end.

  47. Write more, thats aall I have to say. Literally, it seems as though you
    relie on thhe video to make your point. You obviously
    know what youre talking about, whhy waste your intelligence on just posting videos to yohr weblog when you could
    be giving us something informative to read?

  48. When I initially commented I appear to have clicked the -Notify me when new
    comments are added- checkbox and from now on whenever a
    comment is added I get 4 emails with the same comment. Perhaps there is a
    way you are able to remove me from that service? Kudos!

Leave a Reply to winter womens coat Cancel reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>