Showing posts with label groovy. Show all posts
Showing posts with label groovy. Show all posts

Saturday, June 25, 2016

Quickly create groovy script in Intellij IDEA

It is sometimes frustrating when you need quickly check db, non-trivial http, or AWS with Groovy. It is fine and great if standard lib is enough for your case, but once you need new libraries, it is not that quick. Grab is amazing, but you need to specify dependency ids and recall exact API and settings. At some point I had a bunch of typical scripts with typical settings, but you have to be in same project to find them, remember not to commit them, and they tend to grow big and have their own lives at some point. Bad for hacking and it is not convenient for majority of one-off cases.

That is where I started using IntelliJ Live Templates: they are available in every project, easily configurable, you can dispose or save results as needed, they accessible from regular files, scratches, even Groovy console.

My typical template consists of everything needed for single use case: grabs, imports, initialisation and bunch of examples. It is always easier to slash unneeded stuff than recall all API specifics. For me, it is more like tldr output, than what typical default template looks like.

There is example of one template I use for AWS S3:


@Grab('com.amazonaws:aws-java-sdk:1.9.31')
import com.amazonaws.auth.BasicAWSCredentials
import com.amazonaws.services.s3.AmazonS3Client
import com.amazonaws.services.s3.model.ObjectMetadata

def client = new AmazonS3Client(new BasicAWSCredentials('access', 'secret'))

println client.listObjects("bucket", "name").getObjectSummaries().collect{it.key}

def meta = new ObjectMetadata()
meta.setContentType("text/plain")
client.putObject("bucket", "name", new ByteArrayInputStream("1".bytes), meta)

client.deleteObject("bucket", "name")

println new String(client.getObject("bucket", "name").objectContent.bytes)

client.listVersions("bucket", "name").versionSummaries.each {
    println "${it.deleteMarker}   ${it.key}   ${it.size}"
}

client.deleteVersion("bucket", "name", "123")

And of course, you can have multiple templates preconfigured for different environments and use cases.

Biggest disadvantages comparing with regular files is that they are stored somewhere deeply in IntelliJ and not easily transferable, still they are easily exportable with other settings. Other disadvantage is that content is not searchable, so you have to pack all keywords as abbreviation or description, but anyway it is not part of universal search.

Saturday, December 5, 2015

Returning first successful result from CompletableFuture

Recently I wanted to get first non empty result from bunch of CompletableFutures. For my case, allOf - is bad because it runs all futures till end and returns nothing. AnyOf - is bad because it returns first result, even if it is empty. So there is no such function in Java API, but it is still very easy to implement, and maybe it is even more logical, because actually there are 2 cases: found or not found; and it can be easily solved with two CompletableFutures. There is example:

import java.util.concurrent.CompletableFuture

success = new CompletableFuture()

def futures = (1..10).collect { CompletableFuture.supplyAsync {
                                    if (val == 6) {
                                        success.complete(val)
                                    } else {
                                        Thread.sleep(1000)
                                    }
                                }
                          }

println CompletableFuture.allOf(futures as CompletableFuture[])
        .thenApply({ throw new RuntimeException("not found") })
        .applyToEither(success, {it}).get()


This way it will return result if there is any, or throw exception if all done and there is no result, and it can be easily adjusted to return default value, for example.

Saturday, November 28, 2015

Groovy framework for AWS lambda

I was playing with AWS lambda recently and I find it pretty exciting. It is cheap, does not require any maintenance and it is simple. The biggest problem I found so far, is lack of tooling. AWS has web UI which does what it supposed to do, but is far from pleasant. AWS command line tools and API are much better, but are AWS centric. So, I have created tool to work with AWS lambda. It uses some convention now, but can be adjusted to whatever feels right for specific situation, idea is that it is just one script with few lines of code. Basically, it does 2 things: uploads application and local testing. I have created small example that illustrates how and what it does.

Example is trivial application that increments number by clicking on button. It is SPA, with static client in S3 and two API functions add and get.

All lambda code is just single Groovy script. It consists of router function, dependencies and logic. Router function is what will be mapped to lambda. It is function that accepts map and returns map. Input map can be plain request JSON, or some additional parameters can be injected at API gateway. Even for this trivial case it was a good idea and I injected configuration parameters - AWS access and secret keys and region for DynamoDB access, so they are hidden at admin level and are environment specific. Simplest way to do routing is just by using some designated parameter, in this example, it is called "function". Doing it with real URI will need AWS configuration, and it is not very convenient and flexible.

In this simple example, router script does logic, but for more complex cases it is better to be just router and logic would be in dependencies.

During packaging Groovy script is compiled and packaged together with all dependencies into uberjar which is required by AWS. After that, it is uploaded via AWS API.

Development can be done and tested locally by using mock server. Mock server mimics AWS, parses input into JSON like lambda, injects parameters like API gateway, includes Access-Control-Allow-Origin and separates configuration via config.js, supports debug and reloading.

Wednesday, September 16, 2015

Using IDE scripting console in Intellij IDEA

Some time ago, Intellij added option to write simple scripts to automate it's functions, extract information and write simple plugins. It is located under Tools / IDE scripting console. It is very basic now, just scripting file that runs inside of Intellij JVM and has access to Intellij API.

 There is simple Groovy example that automatically adds non-suspending breakpoint with log expression:

import com.intellij.openapi.vfs.LocalFileSystem
import com.intellij.openapi.fileEditor.FileDocumentManager
import com.intellij.openapi.project.ProjectManager
import com.intellij.xdebugger.breakpoints.SuspendPolicy

def file = LocalFileSystem.getInstance().findFileByPath("/users/test/project/src/Test.java")
def doc = FileDocumentManager.getInstance().getDocument(file)
def bp = ProjectManager.getInstance().getOpenProjects()[0].getComponent("DebuggerManager").getBreakpointManager().addLineBreakpoint(doc, 11)
bp.getXBreakpoint().setLogExpression('"test"')
bp.getXBreakpoint().setSuspendPolicy(SuspendPolicy.NONE)


Of course, this basic task can be done through XML configuration, but IDE scripting has access to much more and can do trickier tasks. There is very cool demo. Generally, from what I have seen, API is pretty impressive, it is intuitive, powerful and easy to start with.

My biggest complains are that it is pretty verbose (as Java itself), it would be great if in future versions it would be solved at least at tooling levels - with auto imports for packages, autocompletion, and more convenience functions, like attaching to actions, getting projects or files quickly. Second problem is total lack of documentation. Basically, what I have found is only GitHub repo of Intellij community edition. It has multiple modules with API scattered across them, most generic classes are under Core API.

Thursday, June 25, 2015

How to create executable JAR from single Groovy script

Groovy scripts are nice - they allow to use dependencies and write logic in small single file, but Groovy is mostly available on developers machine, but not on servers, so deploying them is a little more tricky. You can package them in uberjar with Gradle, but it requires additional build file and compatible structure, so it is more like project than script already.

To keep things simple, I have created Groovy script that converts other Groovy scripts into executable JARs. It packages all dependencies from Grab and compiles original script into Java main class with same name as script and sets it in manifest. So it is perfectly usable from empty JVM. JAR structure is also compatible with AWS lambda, so it can be launched from there too, just put logic in specific method. To convert your script, just download it from here and run like:

./scriptjar.groovy input.groovy output.jar

Script tries to find groovy libs via GROOVY_HOME, so set it or just hack it for your location, it is just one small Groovy script.

Wednesday, November 5, 2014

JSON converter is not working in unit tests in Grails

If calling grails.converters.JSON from unit test throws error like:

java.lang.RuntimeException: org.codehaus.groovy.grails.web.converters.exceptions.ConverterException: Unconvertable Object of class: my.blog.Post
	at org.codehaus.groovy.grails.web.converters.AbstractConverter.toString(AbstractConverter.java:111)
	at org.codehaus.groovy.runtime.InvokerHelper.format(InvokerHelper.java:616)
	at org.codehaus.groovy.runtime.InvokerHelper.format(InvokerHelper.java:559)


It is easy to fix this error by adding ControllerUnitTestMixin to the test definition like:

@TestMixin(ControllerUnitTestMixin)
@TestFor(BlogService)
class BlogServiceTests {
...


Friday, October 31, 2014

Grails requires restart after changes in domain class

Grails officially supports domain class reloading since version 2, but recently I found problems using it together with MongoDB GORM plugin, because it required application restart after every change to domain class.
So if Hibernate is primary GORM and Mongo is used only for some particular cases, disabling it can be a good option to make development easier.

Friday, October 17, 2014

Adding common methods to Grails domain objects

If you don't want to change your domain hierarchy, but need to add common function to multiple domains without copy/paste, one simple option is to create and implement trait. For example:
class Test implements LogTrait {
  String name
}

trait LogTrait {
  def springSecurityService

  def afterInsert() {
    log.info(this as JSON)
  }
}
Nice bonus is that events and injected services work just fine as it would be in original class.


Friday, September 5, 2014

Script that logs time to JIRA from Git commit messages

If you need to report work time in JIRA and native comment hashes are not suitable for you, there is simple Groovy script that does the same and can be adjusted to particular scenario or configuration:

@GrabResolver(name='test', root='https://maven.atlassian.com/content/groups/public/')
@Grab( group ='com.atlassian.jira.plugins', module = 'jira-soapclient', version = '4.4' )
@Grab(group='org.eclipse.jgit', module='org.eclipse.jgit', version='3.3.2.201404171909-r')

import com.atlassian.jira.rpc.soap.client.JiraSoapServiceServiceLocator
import com.atlassian.jira.rpc.soap.client.RemoteWorklog
import groovy.transform.TailRecursive
import org.eclipse.jgit.lib.Repository
import org.eclipse.jgit.revwalk.RevCommit
import org.eclipse.jgit.revwalk.RevWalk
import org.eclipse.jgit.storage.file.FileRepositoryBuilder
import java.text.SimpleDateFormat
import static java.util.Calendar.*

@TailRecursive
static Set<RevCommit> subcommits(RevCommit commit, int start, RevWalk walk, def all) {
  if (commit.commitTime > start) {
    all << commit
    commit.parents.collect{ walk.parseCommit(it) }.each {
      subcommits(it, start, walk, all)
    }
  }
  return all
}

Repository repository = new FileRepositoryBuilder().setGitDir(new File(args[3])).setMustExist(true).build()
RevWalk walk = new RevWalk(repository)

def formattime = new SimpleDateFormat('yyyy-MM-dd')
formattime.setTimeZone(TimeZone.getTimeZone("UTC"))
Date start = formattime.parse(args[4])
Date end = formattime.parse(args[5])

def all = subcommits(walk.parseCommit(repository.resolve("HEAD")), start.time / 1000L as int, walk, new HashSet())
def mine = all.findAll{ it.authorIdent.name == args[6] && it.fullMessage =~ /^PRJ-[0-9]+/ }

def actual = mine.min{ it.commitTime }

def tasks = (start..end).findAll{!(it[DAY_OF_WEEK] in [SUNDAY, SATURDAY])}.collect {date ->
  actual = mine.find{ date.time / 1000L <= it.commitTime && (date + 1).time / 1000L > it.commitTime } ?: actual
  [date:date, task:(actual.fullMessage =~ /(PRJ-[0-9]+)/)[0][1]]
}

if (args[0] == 'test') {
  tasks.each { println "${it.date} - ${it.task}" }
} else if (args[0] == 'prod') {
  def jiraSoapService = new JiraSoapServiceServiceLocator().getJirasoapserviceV2(new URL('http://jira.com/rpc/soap/jirasoapservice-v2'))
  String token = jiraSoapService.login(args[1], args[2])
  tasks.each { jiraSoapService.addWorklogAndRetainRemainingEstimate(token, it.task, new RemoteWorklog(timeSpent: "8h", startDate:it.date.toCalendar())) }
}



It uses a bunch of command line parameters - first is either prod or test; test just prints tasks, prod sends them to JIRA. There is an example CLI command:

groovy gittotempo.groovy prod jirauser jirapassword c:\work\project\.git 2014-08-01 2014-08-31 gituser

Friday, August 15, 2014

Disable pretty printing of JSON in Grails

It is easy to print any object as JSON in Grails
[key:'value'] as JSON

But it gives human readable, nice, multiline formatted version of JSON with indents. If that is not needed, it is easy to switch it off, like:
new JSON([key:'value']).toString(true)
Much uglier, but does the job.

Thursday, May 29, 2014

Grabing JavaScript libraries via WebJars in Spring Boot

WebJars is nice and familiar way to work with JavaScript dependencies from Java projects. Spring Boot supports it, and it is pretty easy to add them for Groovy projects, just with good old Grab annotation, like:

@Grab("org.webjars:canjs:2.0.2")

Then all CanJS files become available under /webjars/canjs/2.0.2/. If not sure, list of files can be looked up at WebJars website.

Wednesday, April 30, 2014

Launching Spring Boot in Groovy with Grab

Since launch of Spring Boot I was wondering why it needs separate CLI, especially for Groovy where there is already Grab. Of course CLI provides some additional features, like reloading and out of the box dependencies, but why no one runs Boot with vanilla Groovy? So I tried it myself, and it looks like problem is that there is conflict with default Groovy libs, because standard installation includes servlet-api-2.4.jar which does not work with current Tomcat or Jetty, and as there is no anything like fork mode in Grails, there is only one way to avoid it - not to load this jar (delete it or use custom load conf).

Otherwise this works fine for me:

@Grab("org.springframework.boot:spring-boot-starter-web:1.0.2.RELEASE")
@Grab("org.springframework.boot:spring-boot-starter-actuator:1.0.2.RELEASE")
import org.springframework.boot.SpringApplication
import org.springframework.boot.autoconfigure.EnableAutoConfiguration
import org.springframework.web.bind.annotation.*

@RestController
@EnableAutoConfiguration
class ThisWillActuallyRun {

  @RequestMapping("/")
  String home() {
    "Hello World!"
  }
}
SpringApplication.run(ThisWillActuallyRun, args)


Tuesday, April 15, 2014

IllegalStateException Method on class was used outside of a Grails application

After upgrade to 2.3 and running Geb functional UI tests, they started to fail with error:

java.lang.IllegalStateException: Method on class [my.Test] was used outside of a Grails application. If running in the context of a test using the mocking API or bootstrap Grails correctly.
       at my.Test.currentGormStaticApi(Test.groovy)
       at my.Test.$static_methodMissing(Test.groovy)
       at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
       at groovy.lang.MetaClassImpl.invokeStaticMissingMethod(MetaClassImpl.java:1372)
       at groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1360)
       at groovy.lang.ExpandoMetaClass.invokeStaticMethod(ExpandoMetaClass.java:1123)
       at org.codehaus.groovy.runtime.callsite.StaticMetaClassSite.call(StaticMetaClassSite.java:50)
       at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
       at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
       at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)


After some pretty painful investigation, I found out that problem is when running tests with new fork mode properties, and is easy to solve by removing these options from BuildConfig.groovy:

grails.project.fork = [
    test: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256, daemon:true],
    run: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256, forkReserve:false],
    war: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256, forkReserve:false],
    console: [maxMemory: 768, minMemory: 64, debug: false, maxPerm: 256]
]

Thursday, April 3, 2014

MeteorJS like application in Grails

MeteorJS is new web framework that combines server-side and client-side programming into one, leaving mostly just client-side. I am defenitely not expert with tool and just seen demo on their homepage, but WOW effect is sure impressive. It suppose to kill all Rails and server-side development in general. Thank god I am full-stack developer, but anyway I am scared.

General idea behind this framework is autogenerating all server-side code, leaving mostly just configuration. Similar feature was recently introduced in Grails, so I was wondering - is it possible to create something similar to MeteorJS in Grails. And there is my experience.

Basically, it needs autogenerated REST, Javascript ORM and server push. REST is Grails feature since version 2.3; for client-side ORM I picked CanJS as it looks most similar to Meteor; for server push in Grails I picked spring-websocket plugin (because it is first result in Google, why else?). Unfortunately, spring-websocket is M1 and it relies on Spring 4, so Grails is 2.4M1 too, so it is not production safe, but MeteorJS is 0.8.0 now too, so we are even.

Application idea is simple TODO app. You can find full source here. There is defenitely some boilerplate code as it is not plugin, but I think in general it is pretty similar to MeteorJS example. It consists of 3 meaningful components: view, controller and domain object.

View is Mustache template, and is pretty straightforward if you are familiar with CanJS:

<h1>test</h1>
<table>
{{#todos}}
  <tr>
    <td>{{description}}</td><td><input type="checkbox" can-value="done"></td>
    <td><input type="button" value="Delete" can-click="delete"></td>
    </tr>
{{/todos}}
</table>

Description
<input type="text" can-value="description">
<input type="button" can-click="add" value="Add">

Controller is CanJS component with ORM definition:

var Todo = can.Model({
  findAll: function(id)   {return $.get('/grails-meteor-example/todos.json/')},
  findOne: function(id)   {return $.get('/grails-meteor-example/todos.json/'+id)},
  create:  function(data) {return $.post('/grails-meteor-example/todos.json/', data)},
  update:  function(id)   {return $.ajax({type: 'PUT',    url: '/grails-meteor-example/todos.json/'+id })},
  destroy: function(id)   {return $.ajax({type: 'DELETE', url: '/grails-meteor-example/todos/'+id, contentType: 'application/json' })}
}, {});

can.Component.extend({
  tag: "todos",
  template: can.view("todos.mustache"),
  scope: {
    todos: new Todo.List({}),
    description: can.compute(''),
    add: function() {
      new Todo({'description':this.description(), 'done':false}).save();
      this.description('');
    },
    delete: function(todo) {
      todo.destroy();
    }
  },
  events: {
    "{todo} change": function(){
      this.scope.attr('todos', new Todo.List({}));
    }
  }
});

$(document).ready(function() {
  $("body").html( can.view.mustache("<todos></todos>"));
});


Domain is Grails domain object mapped as REST controller:

package org.grmeteor

import grails.rest.Resource

@Resource(uri='/todos')
class Todo {
  String description
  boolean done
}


View is totally similar to MeteorJS. Controller is similar, but also contains ORM definition and server push event. Both could be easily autogenerated with Grails plugin, but at least push can be also left for practical applications (probably, you will not want to have notification with millions of parallel users). Domain is pure Grails overhead, but it would probably still be needed for SQL backends in MeteorJS (and hey, those server-side devs want to eat too!).

Of course there are other files too, like server push sender and client side HTML container for CanJS, but these are mostly static and sure can be easily hidden by plugin. So only part that is missing to make Grails MeteorJS-cool is plugin itself, which even if not for practical cases could be nice to have for WOW effect and MVPs.

Wednesday, April 2, 2014

Grails REST redirect after DELETE with jQuery

If you do delete from AJAX request (directly or via framework), sometimes Grails REST controller redirects response to index method, which when done with DELETE method leads to 405 HTTP error.
This happens because of incorrect content type in request. To fix it, contentType needs to be explicitly set in request, like:

$.ajax({
    url: '/todos/'+id,
    type: 'DELETE',
    contentType: 'application/json'
  })

Tuesday, February 11, 2014

Grails dies in the middle of Jenkins jobs after update

If you have upgraded recently and facing problem when job suddenly dies in the middle of test execution, like:

19:20:39  | Running 277 unit tests... 156 of 277
19:20:53  Build step 'Execute shell' marked build as failure
19:20:54  Description set: 

Most probably it is because since version 2.2 test-app, run-app, run-war runs in fork mode and Jenkins has some fancy zombie process detection that kills spawned processes it does not like.
Quick solution is to disable fork in test environment or you can refer my previous post about hiding such processes.

Monday, February 3, 2014

No such property error in Cucumber specs with Geb

It is common to write Geb specs directly referencing page elements like:
  to LoginPage
  loginField << login
  passwordField << password
  loginButton.click()

But if you try doing same with Cucumber step files, you will get exception, like:
No such property: loginField for class: steps.Login_steps
groovy.lang.MissingPropertyException: No such property: loginField for class: steps.Login_steps
 at steps.Login_steps$_run_closure2.doCall(Login_steps.groovy:42)
 at ✽.When user enters login and password(Login.feature:7)

This is because page object is not bound and you should reference it explicitly, like:
  to LoginPage
  page.loginField << login
  page.passwordField << password
  page.loginButton.click()


Friday, January 31, 2014

Sending ActiveMQ messages from Groovy

Didn't found it in Google, so there is quick example how to send JMS message from Groovy script using Grab:


@Grab(group='org.apache.activemq',module = 'activemq-all', version='5.5.0')

import org.apache.activemq.ActiveMQConnectionFactory
import javax.jms.Session

new ActiveMQConnectionFactory(brokerURL: 'tcp://localhost:61616').createConnection().with {
  start()
  createSession(false, Session.AUTO_ACKNOWLEDGE).with {
    createProducer().send(createQueue("queue"), createTextMessage("test"))
  }
  close()
}


Friday, December 20, 2013

Access to services from gsp views in Grails

Accessing services from views is considered bad practice, so there is no easy way to do it. It defenitely should not be overused, as it complicates testing and couples logic with presentation. However it is not necessary crime and can be handy for some cases.

For example, if you need access to user parameters. Traditional way to do it, is by using taglibs. What is less known is that you can use taglibs not just to produce text output, but also inside snippets of code in views. For example, you can use it to select users preferred language without need to inject it into every controller:

Taglib:

class UserSettingsTagLib {
    def springSecurityService

    def myLanguage = { attrs, body ->
      out << springSecurityService.currentUser?.myLanguage?.id
    }

}


Gsp:

<g:select name="language" optionKey="id" optionValue="name" value="${g.myLanguage()}" from="${Language.list().sort{it.name}}"/>


Thursday, November 21, 2013

Script to parse groovy source code

Recently, I had to parse Groovy class to extract some information. Reflection was not good as it was not easy to get all dependencies and I had to preserve comments too. This is quite easy to do by utilizing Groovydoc internals which are part of the default libraries:


import antlr.collections.AST
import org.codehaus.groovy.antlr.AntlrASTProcessor
import org.codehaus.groovy.antlr.SourceBuffer
import org.codehaus.groovy.antlr.UnicodeEscapingReader
import org.codehaus.groovy.antlr.parser.GroovyLexer
import org.codehaus.groovy.antlr.parser.GroovyRecognizer
import org.codehaus.groovy.antlr.treewalker.SourceCodeTraversal
import org.codehaus.groovy.tools.groovydoc.SimpleGroovyClassDoc
import org.codehaus.groovy.tools.groovydoc.SimpleGroovyClassDocAssembler

def reader = new File("/path/to/package/org/Groovy.groovy").newReader()
SourceBuffer sourceBuffer = new SourceBuffer()
UnicodeEscapingReader unicodeReader = new UnicodeEscapingReader(reader, sourceBuffer)
GroovyLexer lexer = new GroovyLexer(unicodeReader)
unicodeReader.setLexer(lexer)
GroovyRecognizer parser = GroovyRecognizer.make(lexer)
parser.setSourceBuffer(sourceBuffer)
parser.compilationUnit()
AST ast = parser.getAST()

def visitor = new SimpleGroovyClassDocAssembler("/path/to/package", "org/Groovy.groovy", sourceBuffer, [], new Properties(), true)
AntlrASTProcessor traverser = new SourceCodeTraversal(visitor)
traverser.process(ast)
SimpleGroovyClassDoc doc = (visitor.getGroovyClassDocs().values() as List)[0]
doc.methods().each {
  println it.name()
  println it.commentText()
  println it.annotations()
}


This is definitely not nicest Groovy code, and looks more like Java, but gets the job done. By the way, it should be possible to parse Java too, though I didn't tried to.