> Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.
CEL is a well specified, reasonably fast "embeddable" language with familiar syntax. I'm sure there are other languages that fits the description though.
I've seen but haven't used CEL. Anybody with experience with competing tech have any strong opinions? I've used OPA, know CEL used by GCP and Kyverno, but otherwise haven't seen anything compelling enough to move away from the OPA ecosystem.
And even then, I'm not sure it's apples to apples, at least if by Rego you're thinking of OPA. CEL and Rego take very different approaches, with CEL being quite procedural, while Rego is about constraint satisfaction, not unlike Prolog. At $WORK, Rego (in the form of OPA) gets used quite a bit for complicated access control logic, while CEL gets used in places where we've simpler logic that needs to be broken out and made configurable, and a more procedural focus works there.
I would love if languages like Scala, Swift or F# had something like Cel but running at compile time so your program was evaluated against those restrictions. I believe a language called Idris has something like this
Are you suggesting to compile CEL into native code and run the compiled code at runtime (i.e. as a predicate function)? I think this is doable and I vaguely remember this was how it's implemented initially.
But most use cases are treating CEL as a user provided config, which requires runtime parsing and execution.
A better solution would be first-class metaprogramming, like in Zig or LISP. Maybe with some subset which guarantees to halt (I.e. no unbounded loops, no recursion, no FFI, known input size, hard time limits, etc.)
That's an anti pattern, at least the way we use it. If you need to add complexity, you define custom functions. If that's not enough, CEL probably isn't the right choice, and you'd be doing yourself no favors banging it into that square hole.
It's not really a configuration language like Jsonnet and CUE. It's an expression language for specifying things like conditions and policies. You _could_ abuse it as a configuration language, but it'd be overkill.
Yup, it's really a good fit for simple constraints eg in IAM systems. Give user X permission to do Y, but subject to some CEL expression like date comparison (auto-expiring grants), resource path prefix or similar.
It seems weird to require an entirely new programming language for this tbh. They make the claim that it is special because it's not Turing-complete, but that's nonsense. Turing completeness is almost never a property that is important. I think in this case they're equating Turing incompleteness with "doesn't take a long time to execute" but that isn't really the case at all.
The property you really want is "can be cancelled after a certain amount of compute time - ideally a deterministic amount", and you can obviously do that with Turing complete languages.
No, they're equating _Turing completeness_ with _might not terminate_. CEL, Expr, Rego, and other languages like them are intended to guarantee to complete. You can't do that cleanly with a Turing complete language.
You can estimate cost of CEL program using static analysis before running it. "estimate" only because size of runtime data is generally unknown (but obv you could limit that).
Query DSLs are designed to simplify query planning by intentionally avoiding certain language features. You have many different choices on how to execute a query - in SQL for example, there's table scans, index seeks/scans, joins, etc. and you can execute them in different order. By being able to analyze the query upfront you can estimate the relative costs of different plans and choose the best one. Less powerful languages result in more predictable estimates because they're simpler to analyze.
> Known constraints like guaranteed to terminate is useful.
"Guaranteed to terminate" actually means "guaranteed to terminate in finite but possibly arbitrarily large time" which is really not a useful property.
There's no practical difference between a filter that might take 1 billion years to run and one that might take more than a billion years.
And your service puts an upper bound on input size and cel expression size. (True for all practical applications.)
You can actually get a guarantee tha t you can't construct a billion year expression. And even guarantee that all expressions will evaluate in let's say 60 secs.
Turing completeness by itself does not guarantee this but it is a necessary prerequisite for these guarantees.
It's not fully applicable here, but industry standard DSLs also stick around because non-programmers find learning it a good investment.
I have a business analytics friend that knows SQL because it's part of his workflows.
But Excel, Notion, Power BI, and other low/no-code tools all have their own data filtering and transformation languages (or dialects). He'd rather spend his time learning more about his line of business, than an aspect of yet another cloud tool that gets forced on him.
What you really want is "can be completed after a certain amount of time", not "can be cancelled". You don't want iam policy rules to be skipped because they took too long.
Correct, but you can also reject filters that will take longer statically. The point is not "any arbitrary CEL program will run in less than 10us", it's that I can encode "do not allow filters that take more than 10us to evaluate" an then have a very high degree of confidence that that will be true for any user provided filter that is accepted (and if I'm wrong it'll be...11us, not 5s)
In the common use-cases for CEL that I've seen, you don't want to skip evaluation and fail open or closed arbitrarily. That can mean things like "abusive user gets access to data they should not be allowed to access because rule evaluation was skipped".
You also may have tons of rules and be evaluating them very often, so speed is important.
Shameless plug, but if you want to try CEL in your browser: https://celq-playground.github.io/
I wrote the playground and I should link to your website in my docs. This is neat.
CEL is useful for any custom computation you want to do on your critical path without having that blow out in a ridiculous fashion.
Yes you can embed other languages however constraining evaluation costs is not a first class feature.
I think people commenting misunderstood what CEL offers.
Remember the famous https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule?
> Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.
CEL is a well specified, reasonably fast "embeddable" language with familiar syntax. I'm sure there are other languages that fits the description though.
I've seen but haven't used CEL. Anybody with experience with competing tech have any strong opinions? I've used OPA, know CEL used by GCP and Kyverno, but otherwise haven't seen anything compelling enough to move away from the OPA ecosystem.
The kubernetes apiserver allows using CEL in CustomResourceDefinition validation rules: - https://kubernetes.io/docs/reference/using-api/cel/ - https://kubernetes.io/docs/tasks/extend-kubernetes/custom-re...
It also allows using CEL in ValidatingAdmissionPolicies: - https://kubernetes.io/docs/reference/access-authn-authz/vali...
I think apples to apples comparison would be comparing against Rego. To me CEL is more appealing due to its simplicity.
And even then, I'm not sure it's apples to apples, at least if by Rego you're thinking of OPA. CEL and Rego take very different approaches, with CEL being quite procedural, while Rego is about constraint satisfaction, not unlike Prolog. At $WORK, Rego (in the form of OPA) gets used quite a bit for complicated access control logic, while CEL gets used in places where we've simpler logic that needs to be broken out and made configurable, and a more procedural focus works there.
Rego is much more powerful, and can do things cel can't.
CEL is much more computationally limited as it aims to keep evaluations in the microsecond range.
With OPA you can easily create policies that take tens, hundreds or even thousands of millisecond.
That comes at the expense of a lot of power though, so much of the complex logic that you can write in OPA simply isn't achievable in CEL.
I would love if languages like Scala, Swift or F# had something like Cel but running at compile time so your program was evaluated against those restrictions. I believe a language called Idris has something like this
Are you suggesting to compile CEL into native code and run the compiled code at runtime (i.e. as a predicate function)? I think this is doable and I vaguely remember this was how it's implemented initially.
But most use cases are treating CEL as a user provided config, which requires runtime parsing and execution.
I was thinking of something like this:
type MyType{ myName: string where size > 8, year: number where number > 2000 }
Then, whenever this type is used, at compile time, an evaluation is done to ensure that the type restrictions are enforced.
A better solution would be first-class metaprogramming, like in Zig or LISP. Maybe with some subset which guarantees to halt (I.e. no unbounded loops, no recursion, no FFI, known input size, hard time limits, etc.)
CEL is used a lot in FHIR as Path Expressions
I did not know that. Path expressions for access control?
Mostly for validating the FHIR Resources themselves
Does CEL have any way to import other files? i.e. could it serve as a general purpose config language like jsonnet?
That's an anti pattern, at least the way we use it. If you need to add complexity, you define custom functions. If that's not enough, CEL probably isn't the right choice, and you'd be doing yourself no favors banging it into that square hole.
It's not really a configuration language like Jsonnet and CUE. It's an expression language for specifying things like conditions and policies. You _could_ abuse it as a configuration language, but it'd be overkill.
Yup, it's really a good fit for simple constraints eg in IAM systems. Give user X permission to do Y, but subject to some CEL expression like date comparison (auto-expiring grants), resource path prefix or similar.
It seems weird to require an entirely new programming language for this tbh. They make the claim that it is special because it's not Turing-complete, but that's nonsense. Turing completeness is almost never a property that is important. I think in this case they're equating Turing incompleteness with "doesn't take a long time to execute" but that isn't really the case at all.
The property you really want is "can be cancelled after a certain amount of compute time - ideally a deterministic amount", and you can obviously do that with Turing complete languages.
No, they're equating _Turing completeness_ with _might not terminate_. CEL, Expr, Rego, and other languages like them are intended to guarantee to complete. You can't do that cleanly with a Turing complete language.
Right but "guaranteed to terminate" is not a useful property. You could write a program that terminates... after a billion years.
You can estimate cost of CEL program using static analysis before running it. "estimate" only because size of runtime data is generally unknown (but obv you could limit that).
"You can" - in theory, or does this actually exist?
https://pkg.go.dev/github.com/google/cel-go/cel#Env.Estimate...
There’s a whole section on this on TFA: https://celbyexample.com/execution-cost/
With certain macros disabled like .map the runtime is O(code length)!
Query DSLs are designed to simplify query planning by intentionally avoiding certain language features. You have many different choices on how to execute a query - in SQL for example, there's table scans, index seeks/scans, joins, etc. and you can execute them in different order. By being able to analyze the query upfront you can estimate the relative costs of different plans and choose the best one. Less powerful languages result in more predictable estimates because they're simpler to analyze.
Ease/ability to embed in other language safely. Predictability of memory, execution. Known constraints like guaranteed to terminate is useful.
no Doom running on cel.
I recently wanted to expose some basic user auto tagging/labeling based on the json data.
I chose cel, over python, SQL because I could just import the runtime in C++, or any language that implements it (python, js etc..)
Safely running a sandboxed python execution engine is significantly more effort and lower performance.
At this cel excels.
Where it didn't was user familiarity and when the json data itself was complex.
> Known constraints like guaranteed to terminate is useful.
"Guaranteed to terminate" actually means "guaranteed to terminate in finite but possibly arbitrarily large time" which is really not a useful property.
There's no practical difference between a filter that might take 1 billion years to run and one that might take more than a billion years.
Yes but when you combine it with the other guarantees on performance.
https://github.com/google/cel-spec/blob/master/doc/langdef.m...
And your service puts an upper bound on input size and cel expression size. (True for all practical applications.)
You can actually get a guarantee tha t you can't construct a billion year expression. And even guarantee that all expressions will evaluate in let's say 60 secs.
Turing completeness by itself does not guarantee this but it is a necessary prerequisite for these guarantees.
> Turing completeness by itself does not guarantee this
They should update their home page then!
It's not fully applicable here, but industry standard DSLs also stick around because non-programmers find learning it a good investment.
I have a business analytics friend that knows SQL because it's part of his workflows.
But Excel, Notion, Power BI, and other low/no-code tools all have their own data filtering and transformation languages (or dialects). He'd rather spend his time learning more about his line of business, than an aspect of yet another cloud tool that gets forced on him.
What you really want is "can be completed after a certain amount of time", not "can be cancelled". You don't want iam policy rules to be skipped because they took too long.
Well CEL doesn't offer that guarantee. For any given "certain amount of time" you can write a CEL filter that takes longer.
Correct, but you can also reject filters that will take longer statically. The point is not "any arbitrary CEL program will run in less than 10us", it's that I can encode "do not allow filters that take more than 10us to evaluate" an then have a very high degree of confidence that that will be true for any user provided filter that is accepted (and if I'm wrong it'll be...11us, not 5s)
In the common use-cases for CEL that I've seen, you don't want to skip evaluation and fail open or closed arbitrarily. That can mean things like "abusive user gets access to data they should not be allowed to access because rule evaluation was skipped".
You also may have tons of rules and be evaluating them very often, so speed is important.
See my other comment - you can refuse to accept CEL filters that take too long to begin with.
Say “halting problem” without saying “halting problem” ;)
There is a practical solution to it called “metering”, like gas mechanism in Ethereum’s EVM or cost calculation for complex GraphQL queries.
Yeah I think it's typically called "fuel".